zishuo/nip
1
0
Fork 0
mirror of https://github.com/cunnie/sslip.io.git synced 2025-10-07 00:23:44 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Dismantle DNS-backed key-value store, k-v.io

Browse Source 
I'm no longer engaged on setting up k-v.io; I thought it'd be cool to
have a DNS-backed etcd implementation, but now I don't care anymore.

There were technical challenges, too: Specifically, updating values did
not play well with DNS caching — you'd get the old value after updating.

If the service became popular, I'd quickly run out of disk space on my
tiny cloud VMs.

The service would most likely be used by people doing data exfiltration
via DNS. I already have enough problems with sslip.io scammers — the
last thing I want is to sign up for dealing with k-v.io scammers.

This commit removes the etcd configuration, certificates, and pipelines.
This commit is contained in:
Brian Cunnie
2024-09-15 07:13:57 -07:00
parent 450fc67a57
commit afe851a867
13 changed files with 0 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
k8s/document_root_k-v.io/index.html
View File

@@ -1,131 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="generator" content="HTML Tidy for HTML5 for Apple macOS version 5.8.0">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="Mark Otto, Jacob Thornton, and Bootstrap contributors">
<meta name="generator" content="Jekyll v3.8.5">
<title>k-v.io</title>
<meta name="description" content="k-v.io">
<meta name="author" content="Brian Cunnie"><!-- cute Green Lock icon -->
<link rel="shortcut icon" type="image/x-svg" href="img/favicon.svg"><!-- Latest
<link rel="canonical" href="https://getbootstrap.com/docs/4.3/examples/starter-template/"><!-- Bootstrap core CSS -->
<link href="https://cdn.jsdelivr.net/npm/bootstrap@4.3.1/dist/css/bootstrap.min.css" rel="stylesheet" integrity=
"sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
<style>
.bd-placeholder-img {
font-size: 1.125rem;
text-anchor: middle;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
@media (min-width: 768px) {
.bd-placeholder-img-lg {
font-size: 3.5rem;
}
}
</style><!-- Custom styles for this template -->
<link href="starter-template.css" rel="stylesheet">
</head>
<body>
<main role="main" class="container">
<div class="starter-template">
<h1>k-v.io</h1>
<p>k-v.io is under construction. I've dismantled the DNS-backed key-value store—no one was using it. Stay
tuned.</p><!--
<p>A DNS-based Key-Value Store. <a href="https://ci.nono.io/teams/main/pipelines/sslip.io"><img alt="badge" src=
"https://ci.nono.io/api/v1/teams/main/pipelines/sslip.io/badge"></a></p>
<h2>Quick Start</h2>
<p>In the following examples, <i>my-key</i> is the key, and <i>my-value</i> is the value.</p>
<ul>
<li>To set a key: <code>dig @ns.sslip.io put.my-value.my-key.k-v.io txt +short</code></li>
<li>To get a key: <code>dig @ns.sslip.io my-key.k-v.io txt +short</code></li>
<li>To delete a key: <code>dig @ns.sslip.io delete.my-key.k-v.io txt +short</code></li>
</ul>
<h3>Notes:</h3>
<ul>
<li>Values are stored as TXT records.</li>
<li>Keys must be valid DNS subdomains, i.e. only alphanumerics and dashes, and cannot exceed 63 characters.
"my-key" is valid, but "my#key" is not.</li>
<li>Values can be one or more subdomains, i.e. alphanumerics, dashes, and <i>dots</i>, and are truncated to 63
characters. For example, <code>dig put.3.14159.pi.k-v.io txt +short</code> sets the the key "pi" to the value
"3.14159".</li>
<li>If you need special characters ("!@#$%^😊"), consider <a href=
"https://en.wikipedia.org/wiki/Base32">Base32</a> encoding, but remember that the padding character ("=") is
illegal.
</li>
<li>The three verbs are <code>put</code>, <code>get</code>, and <code>delete</code>, e.g. <code>dig
delete.pi.k-v.io txt +short</code>.</li>
<li>The verb <code>get</code> is the default verb: <code>dig pi.k-v.io txt +short</code> is identical to
<code>dig get.pi.k-v.io txt +short</code>.</li>
<li>There's no read security: when you set a key "my-super-secret-password" to the value "dont-tell-anyone",
anyone can read it.</li>
<li>There's no write security: you may set the key "best-rugby-team" to the value "all-blacks", and someone
else could change it a minute later to "springboks".</li>
<li>If you don't want someone to mess with your keys, you should probably use GUIDs or something fairly unique
as a key, e.g. <code>dig put.my-value.df616686-26e8-4da8-8104-a24aa0196bc7.k-v.io txt +short</code></li>
<li>I have no idea how well this system will scale.</li>
<li>I'll probably expire keys that haven't been accessed in a month, but I haven't written that code yet.</li>
<li>This project is very much in beta. Assume you may lose your keys at any time.</li>
</ul>
<h3>DNS Caching/Propagation:</h3>
<p>You don't need to specify our nameservers ("<code>@ns.sslip.io</code>") in your <code>dig</code> invocations
if you don't mind dealing with the vagaries of DNS caching and propagation. Here are some of the problems you'll
face:</p>
<ul>
<li>When you change or delete a key, it may take up to 3 minutes for the change to propagate due to DNS
caching.</li>
<li>DNS propagation/caching is particularly troublesome when doing multiple write operations on a key within a
3-minute period. The first one takes effect, the second one doesn't because it's cached at the upstream
nameservers & never reaches the <i>k-v.io</i> nameservers.</li>
</ul>
<h3>Technical Notes:</h3>
<ul>
<li>There are three servers that back this service: two in the USA, one in Singapore, on three different
platforms: AWS, Azure, and Google Cloud.</li>
<li>The underlying key-value store is an etcd cluster. I chose etcd for no particular reason other than it's
what Kubernetes uses.</li>
<li>The <i>k-v.io</i> source code is in Golang, is hosted at <a href=
"https://github.com/cunnie/sslip.io">GitHub</a>, and is licensed under the Apache 2.0 license.
</li>
<li>Much of the deployment information (terraform files, Kubernetes manifests) is also freely available at a
<a href="https://github.com/cunnie/deployments/tree/main/terraform/gcp/gke">different GitHub repo</a>, and some
of the workstation configuration is at a <a href=
"https://github.com/cunnie/bin/blob/main/install_ns-aws.sh">third GitHub repo</a>. Yes, it's an organizational
challenge.
</li>
<li>The DNS server code is a mash-up of key-value store and <a href="https://sslip.io">mapping hostnames with
embedded IP addresses to those addresses</a> (e.g. <code>127.0.0.1.sslip.io</code> → <code>127.0.0.1</code>).
In a perfect world, I'd have separated the codebase into two and deployed <i>k-v.io</i> on a second set of
three nameservers; however, I had neither the time nor the inclination to manage a 2nd set of DNS servers. So
here we are.
</li>
<li>If you have suggestions (or notice that something's broken), please open a <a href=
"https://github.com/cunnie/sslip.io/issues/new/choose">GitHub issue</a>.
</li>
<li>I was motivated to create this service because at my job we use S3 as a key-value store, and getting at the
keys is a challenge because we need the AWS credentials, and to get at those credentials we need to use CredHub
(a Vault-like secrets store). It's a lot of hoops to jump through to find out something as innocuous as the
version number of the latest candidate build.</li>
<li>
<a href="https://dnskv.com/">dnskv.com</a> is a similar service with a rich set of options.
</li>
</ul>
</div>
-->
</div>
</main><!-- /.container -->
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity=
"sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<script>
window.jQuery || document.write('<script src="/docs/4.3/assets/js/vendor/jquery-slim.min.js"><\/script>')
</script>
<script src="/docs/4.3/dist/js/bootstrap.bundle.min.js" integrity=
"sha384-xrRywqdh3PHs8keKZN+8zzc5TX0GRTLCcmivcbNJWm2rs5C8PRhcEn3czEjhAO9o" crossorigin="anonymous"></script>
</body>
</html>