zishuo/nip
1
0
Fork 0
mirror of https://github.com/cunnie/sslip.io.git synced 2025-09-28 04:22:19 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Dismantle DNS-backed key-value store, k-v.io

Browse Source 
I'm no longer engaged on setting up k-v.io; I thought it'd be cool to
have a DNS-backed etcd implementation, but now I don't care anymore.

There were technical challenges, too: Specifically, updating values did
not play well with DNS caching — you'd get the old value after updating.

If the service became popular, I'd quickly run out of disk space on my
tiny cloud VMs.

The service would most likely be used by people doing data exfiltration
via DNS. I already have enough problems with sslip.io scammers — the
last thing I want is to sign up for dealing with k-v.io scammers.

This commit removes the etcd configuration, certificates, and pipelines.
This commit is contained in:
Brian Cunnie
2024-09-15 07:13:57 -07:00
parent 450fc67a57
commit afe851a867
13 changed files with 0 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
ci/pipeline-sslip.io.yml
View File

@@ -65,34 +65,6 @@ jobs:
params:
image: image/image
- name: build-and-push-k-v.io-nginx
plan:
- get: sslip.io-release
trigger: true
- get: sslip.io
- task: build-task-image
privileged: true
config:
platform: linux
image_resource:
type: registry-image
source:
repository: concourse/oci-build-task
inputs:
- name: sslip.io
outputs:
- name: image
params:
CONTEXT: sslip.io/k8s/
DOCKERFILE: sslip.io/k8s/Dockerfile-k-v.io-nginx
IMAGE_PLATFORM: linux/arm64,linux/amd64
OUTPUT_OCI: true
run:
path: build
- put: k-v.io-nginx
params:
image: image/image
- name: build-and-push-sslip.io-nginx
plan:
- get: sslip.io-release
@@ -143,16 +115,6 @@ resources:
source:
uri: https://github.com/cunnie/sslip.io.git
# Where we will push the k-v.io nginx Docker image with HTML assets
- name: k-v.io-nginx
type: registry-image
icon: docker
source:
repository: cunnie/k-v.io-nginx
username: cunnie
password: ((docker_token))
tag: latest
# Where we will push the sslip.io nginx Docker image with HTML assets
- name: sslip.io-nginx
type: registry-image

2
etcd/.gitignore vendored
View File

@@ -1,2 +0,0 @@
*-key.pem
*.csr

193
etcd/README.md
View File

@@ -1,193 +0,0 @@
### Setting Up `etcd`
We set up `etcd` as a backing database for our `sslip.io` webserver.
#### Generate Certificates
We need to generate certificates for our etcd cluster (our cluster will
communicate over TLS, but our clients won't).
- `ca-config.json`. We set the certificates it issues to expire in 30
years (262800 hours) because we don't want to go through a certificate
rotation. Trust me on this one.
- `ca-csr.json`. Again, 30 years.
```shell
cfssl gencert -initca ca-csr.json | cfssljson -bare etcd-ca
```
The key is saved in LastPass as `etcd-ca-key.pem`.
Let's use our newly-created CA to generate the etcd certificates. Note
that we throw almost every IP address/hostname we can think of into the
SANs field (why not?):
```shell
GKE_NODE_PUBLIC_IPv4=$(gcloud compute instances list --format=json |
jq -r '[.[].networkInterfaces[0].accessConfigs[0].natIP] | join(",")')
PUBLIC_HOSTNAMES=ns-aws.sslip.io,ns-azure.sslip.io,ns-gce.sslip.io
HOSTNAMES=ns-aws,ns-azure,ns-gce
IPv4=127.0.0.1,52.0.56.137,52.187.42.158,104.155.144.4,$GKE_NODE_PUBLIC_IPv4
IPv6=::1,2600:1f18:aaf:6900::a
cfssl gencert \
-ca=ca.pem \
-ca-key=etcd-ca-key.pem \
-config=ca-config.json \
-hostname=${PUBLIC_HOSTNAMES},${HOSTNAMES},${IPv4},${IPv6} \
-profile=etcd \
etcd-csr.json | cfssljson -bare etcd
```
The key is saved in LastPass as `etcd-key.pem`.
#### Generating a New Cert for a New etcd Node
Let's say you've introduced _new_ IPv4 addresses, or that you've recreated your
GKE clusters, and all the addresses have changed, then you'll need to
regenerate the certificates:
```
lpass show --note etcd-ca-key.pem > etcd-ca-key.pem
lpass show --note etcd-key.pem > etcd-key.pem
GKE_NODE_PUBLIC_IPv4=$(gcloud compute instances list --format=json |
jq -r '[.[].networkInterfaces[0].accessConfigs[0].natIP] | join(",")')
PUBLIC_HOSTNAMES=ns-aws.sslip.io,ns-azure.sslip.io,ns-gce.sslip.io
HOSTNAMES=ns-aws,ns-azure,ns-gce
IPv4=127.0.0.1,52.0.56.137,52.187.42.158,104.155.144.4,$GKE_NODE_PUBLIC_IPv4
IPv6=::1,2600:1f18:aaf:6900::a
cfssl gencsr \
-key=etcd-key.pem \
-hostname=${PUBLIC_HOSTNAMES},${HOSTNAMES},${IPv4},${IPv6} \
-cert=etcd.pem | cfssljson -bare etcd
cfssl sign \
-ca=ca.pem \
-ca-key=etcd-ca-key.pem \
-config=ca-config.json \
-profile=etcd \
etcd.csr | cfssljson -bare etcd
```
#### Configure ns-aws.sslip.io & ns-azure.sslip.io
Now let's set up etcd on either ns-aws or ns-azure:
```shell
sudo mkdir /etc/etcd # default's okay: root:root 755
IAAS=${HOST/ns-/}
cd /etc/etcd
sudo curl -OL https://raw.githubusercontent.com/cunnie/sslip.io/main/etcd/ca.pem
sudo curl -OL https://raw.githubusercontent.com/cunnie/sslip.io/main/etcd/etcd.pem
sudo curl -o /etc/default/etcd -L https://raw.githubusercontent.com/cunnie/sslip.io/main/etcd/etcd-$IAAS.conf
lpass login brian.cunnie@gmail.com --trust
lpass show --note etcd-key.pem | sudo tee etcd-key.pem
sudo chmod 400 *key*
sudo chown etcd:etcd *key*
```
Let's fire up etcd:
```shell
sudo systemctl daemon-reload
sudo systemctl enable etcd
sudo systemctl stop etcd
sudo systemctl start etcd
sudo journalctl -xefu etcd # look for any errors on startup
sudo systemctl restart sslip.io-dns
dig @localhost metrics.status.sslip.io txt +short | grep "Key-value store:" # should be "etcd"
```
If the messages look innocuous (ignore "serving client traffic insecurely; this
is strongly discouraged!").
Check the cluster:
```shell
export ETCDCTL_API=3
etcdctl member list # first time: "8e9e05c52164694d, started, default, http://localhost:2380, http://localhost:2379, false"
# existing cluster:
660f0ebfd9c21a95: name=ns-aws peerURLs=https://ns-aws.sslip.io:2380 clientURLs=http://localhost:2379 isLeader=true
6e7e4616e1032417: name=ns-azure peerURLs=https://ns-azure.sslip.io:2380 clientURLs=http://localhost:2379 isLeader=false
b77b5c23840fa42b: name=ns-gce peerURLs=https://ns-gce.sslip.io:2380 clientURLs= isLeader=false
```
### Wiping old data
ns-aws & ns-azure:
```
sudo systemctl stop etcd
sudo rm -rf /var/lib/etcd/default/member
sudo systemctl start etcd
```
### Deleting and Re-adding ns-azure
This needs to be done when, for example, ns-azure is rebuilt from scratch.
```bash
ssh ns-aws
export ETCDCTL_API=3
etcdctl member list
# 6e7e4616e1032417: name=ns-azure peerURLs=https://ns-azure.sslip.io:2380 clientURLs=http://localhost:2379 isLeader=false
etcdctl member remove 6e7e4616e1032417
etcdctl member add ns-azure --peer-urls=https://ns-azure.sslip.io:2380
exit
ssh ns-azure
sudo systemctl stop etcd
sudo rm -rf /var/lib/etcd/default/member
sudo -E nvim /etc/default/etcd
# ETCD_INITIAL_CLUSTER_STATE="existing"
sudo systemctl start etcd
etcdctl member list
sudo du -sH /var/lib/etcd/default/member
```
### Updating the GKE PEM
This needs to be done every darn time the nodes are upgraded (there _must_ be a better way)
```bash
cd etcd/
lpass show --note etcd-ca-key.pem > etcd-ca-key.pem
lpass show --note etcd-key.pem > etcd-key.pem
GKE_NODE_PUBLIC_IPv4=$(gcloud compute instances list --format=json |
jq -r '[.[].networkInterfaces[0].accessConfigs[0].natIP] | join(",")')
PUBLIC_HOSTNAMES=ns-aws.sslip.io,ns-azure.sslip.io,ns-gce.sslip.io
HOSTNAMES=ns-aws,ns-azure,ns-gce
IPv4=127.0.0.1,52.0.56.137,52.187.42.158,104.155.144.4,$GKE_NODE_PUBLIC_IPv4
IPv6=::1,2600:1f18:aaf:6900::a
cfssl gencsr \
-key=etcd-key.pem \
-hostname=${PUBLIC_HOSTNAMES},${HOSTNAMES},${IPv4},${IPv6} \
-cert=etcd.pem | cfssljson -bare etcd
cfssl sign \
-ca=ca.pem \
-ca-key=etcd-ca-key.pem \
-config=ca-config.json \
-profile=etcd \
etcd.csr | cfssljson -bare etcd
kubectl delete secret etcd-peer-tls
kubectl create secret generic etcd-peer-tls \
--from-file=ca.pem=<(curl -L https://raw.githubusercontent.com/cunnie/sslip.io/main/etcd/ca.pem) \
--from-file=etcd.pem=etcd.pem \
--from-file=etcd-key.pem=<(lpass show --note etcd-key.pem)
kubectl get secret etcd-peer-tls -o json | \
jq -r '.data."etcd.pem"' | \
base64 -d | \
openssl x509 -noout -text
kubectl rollout restart deployment/k-v.io
sleep 60 && kubectl rollout restart deployment/sslip.io # give time for etcd to come up
git status
git add -p
git ci -m"Update GKE node public IP addrs (etcd.pem)"
git push
```
### Troubleshooting
If `sudo journalctl -xefu etcd` errors with `member xxx has already been
bootstrapped`, then edit `/etc/default/etcd` and set
`ETCD_INITIAL_CLUSTER_STATE="existing"` (previously was `"new"`).

18
etcd/ca-config.json
View File

@@ -1,18 +0,0 @@
{
"signing": {
"default": {
"expiry": "262800h"
},
"profiles": {
"etcd": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "262800h"
}
}
}
}

20
etcd/ca-csr.json
View File

@@ -1,20 +0,0 @@
{
"CA": {
"expiry": "262800h"
},
"CN": "etcd",
"key": {
"algo": "ecdsa",
"size": 256
},
"expires": "2054-02-16T23:59:59Z",
"names": [
{
"C": "US",
"L": "San Francisco",
"O": "etcd",
"OU": "nono.io",
"ST": "California"
}
]
}

14
etcd/ca.pem
View File

@@ -1,14 +0,0 @@
-----BEGIN CERTIFICATE-----
MIICGzCCAcCgAwIBAgIULIX6nw9giY3XwSjQMvjg/A+uhr8wCgYIKoZIzj0EAwIw
ajELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExFjAUBgNVBAcTDVNh
biBGcmFuY2lzY28xDTALBgNVBAoTBGV0Y2QxEDAOBgNVBAsTB25vbm8uaW8xDTAL
BgNVBAMTBGV0Y2QwIBcNMjExMjMxMjExODAwWhgPMjA1MTEyMjQyMTE4MDBaMGox
CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1TYW4g
RnJhbmNpc2NvMQ0wCwYDVQQKEwRldGNkMRAwDgYDVQQLEwdub25vLmlvMQ0wCwYD
VQQDEwRldGNkMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEfSScCTvWM/XcQ4Ab
+jqdPBh35f+xXhukuhW84gH8EVB3fzuWakbw9v6VYOFj5nFlkCLPWYUZiFRjZ0A9
IkXE8KNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0O
BBYEFJJlnHiaL/LTE/NblNi/cj1yorbBMAoGCCqGSM49BAMCA0kAMEYCIQCqSQFS
wKBn9EB4dOK7lxB568U7TK7astH7p+JoFEwR+gIhAM72R7sa55+7RU7RBsCh1PKV
GsSinJbCP8+17wqzqrt3
-----END CERTIFICATE-----

51
etcd/etcd-aws.conf
View File

@@ -1,51 +0,0 @@
# [member]
ETCD_NAME=ns-aws
ETCD_DATA_DIR="/var/lib/etcd/default"
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://ns-aws.sslip.io:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="ns-aws=https://ns-aws.sslip.io:2380,ns-azure=https://ns-azure.sslip.io:2380,ns-gce=https://ns-gce.sslip.io:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
ETCD_CERT_FILE="/etc/etcd/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/etcd-key.pem"
#ETCD_CLIENT_CERT_AUTH="false"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ca.pem"
#ETCD_AUTO_TLS="false"
ETCD_PEER_CERT_FILE="/etc/etcd/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ca.pem"
#ETCD_PEER_AUTO_TLS="false"
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""

51
etcd/etcd-azure.conf
View File

@@ -1,51 +0,0 @@
# [member]
ETCD_NAME=ns-azure
ETCD_DATA_DIR="/var/lib/etcd/default"
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://ns-azure.sslip.io:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="ns-aws=https://ns-aws.sslip.io:2380,ns-azure=https://ns-azure.sslip.io:2380,ns-gce=https://ns-gce.sslip.io:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
ETCD_CERT_FILE="/etc/etcd/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/etcd-key.pem"
#ETCD_CLIENT_CERT_AUTH="false"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ca.pem"
#ETCD_AUTO_TLS="false"
ETCD_PEER_CERT_FILE="/etc/etcd/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ca.pem"
#ETCD_PEER_AUTO_TLS="false"
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""

16
etcd/etcd-csr.json
View File

@@ -1,16 +0,0 @@
{
"CN": "etcd",
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"L": "San Francisco",
"O": "etcd",
"OU": "nono.io",
"ST": "California"
}
]
}

51
etcd/etcd.conf
View File

@@ -1,51 +0,0 @@
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
#ETCD_INITIAL_CLUSTER="default=http://localhost:2380"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_AUTO_TLS="false"
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#ETCD_PEER_AUTO_TLS="false"
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""

18
etcd/etcd.pem
View File

@@ -1,18 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIC5TCCAougAwIBAgIUfteCMiTGziRpdo02Ycq1ccPti0IwCgYIKoZIzj0EAwIw
ajELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExFjAUBgNVBAcTDVNh
biBGcmFuY2lzY28xDTALBgNVBAoTBGV0Y2QxEDAOBgNVBAsTB25vbm8uaW8xDTAL
BgNVBAMTBGV0Y2QwIBcNMjMwODEyMTkxMjAwWhgPMjA1MzA4MDQxOTEyMDBaMGox
CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1TYW4g
RnJhbmNpc2NvMQ0wCwYDVQQKEwRldGNkMRAwDgYDVQQLEwdub25vLmlvMQ0wCwYD
VQQDEwRldGNkMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE9n3v0f+CsUAS0spI
Hhsd/hnVoS0oyONpe5ow/zSKSdM6F0e0T1W9ZDMkfy/QyDOmSSza9Sfz0DqDLkly
xObn8qOCAQswggEHMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcD
AQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUKeZ6GH6hJhzaJOQh
GyhMfQ4zD2owgagGA1UdEQSBoDCBnYIPbnMtYXdzLnNzbGlwLmlvghFucy1henVy
ZS5zc2xpcC5pb4IPbnMtZ2NlLnNzbGlwLmlvggZucy1hd3OCCG5zLWF6dXJlggZu
cy1nY2WHBH8AAAGHBDQAOImHBDS7Kp6HBGibkASHBJKUasGHBGjFqPKHBCJ6V82H
EAAAAAAAAAAAAAAAAAAAAAGHECYAHxgKr2kAAAAAAAAAAAowCgYIKoZIzj0EAwID
SAAwRQIhANiwGftxXMd6MC0YTjUxFv+c05oZ9ANvX+4FY2hdnJX4AiBka23oqwEb
dYK6BnYLMs+72VysLZ+WvOxSKo4rZsJntw==
-----END CERTIFICATE-----

43
k8s/Dockerfile-k-v.io-nginx
View File

@@ -1,43 +0,0 @@
#
# cunnie/sslip.io-nginx
#
# k-v.io nginx Dockerfile
#
# Dockerfile of an nginx server that serves the web
# pages of the k-v.io domain.
#
# Typical start command:
#
# docker run --rm -p 8080:80 cunnie/k-v.io-nginx
#
# To test from host:
#
# curl -I http://localhost:8080
#
FROM fedora AS sslip.io-nginx
LABEL org.opencontainers.image.authors="Brian Cunnie <brian.cunnie@gmail.com>"
RUN dnf install -y \
bind-utils \
iproute \
less \
lsof \
neovim \
net-tools \
nginx \
nmap-ncat \
procps-ng
RUN mv /usr/share/nginx/html /usr/share/nginx/html-orig
COPY document_root_k-v.io /usr/share/nginx/html
ENTRYPOINT [ "/usr/sbin/nginx", "-g", "daemon off;" ]
# for testing:
# ENTRYPOINT /bin/bash
# nginx listens on port 80
# The `EXPOSE` directive doesn't do much in our case. We use it for documentation.
EXPOSE 80/tcp

131
k8s/document_root_k-v.io/index.html
View File

@@ -1,131 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="generator" content="HTML Tidy for HTML5 for Apple macOS version 5.8.0">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="Mark Otto, Jacob Thornton, and Bootstrap contributors">
<meta name="generator" content="Jekyll v3.8.5">
<title>k-v.io</title>
<meta name="description" content="k-v.io">
<meta name="author" content="Brian Cunnie"><!-- cute Green Lock icon -->
<link rel="shortcut icon" type="image/x-svg" href="img/favicon.svg"><!-- Latest
<link rel="canonical" href="https://getbootstrap.com/docs/4.3/examples/starter-template/"><!-- Bootstrap core CSS -->
<link href="https://cdn.jsdelivr.net/npm/bootstrap@4.3.1/dist/css/bootstrap.min.css" rel="stylesheet" integrity=
"sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
<style>
.bd-placeholder-img {
font-size: 1.125rem;
text-anchor: middle;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
@media (min-width: 768px) {
.bd-placeholder-img-lg {
font-size: 3.5rem;
}
}
</style><!-- Custom styles for this template -->
<link href="starter-template.css" rel="stylesheet">
</head>
<body>
<main role="main" class="container">
<div class="starter-template">
<h1>k-v.io</h1>
<p>k-v.io is under construction. I've dismantled the DNS-backed key-value store—no one was using it. Stay
tuned.</p><!--
<p>A DNS-based Key-Value Store. <a href="https://ci.nono.io/teams/main/pipelines/sslip.io"><img alt="badge" src=
"https://ci.nono.io/api/v1/teams/main/pipelines/sslip.io/badge"></a></p>
<h2>Quick Start</h2>
<p>In the following examples, <i>my-key</i> is the key, and <i>my-value</i> is the value.</p>
<ul>
<li>To set a key: <code>dig @ns.sslip.io put.my-value.my-key.k-v.io txt +short</code></li>
<li>To get a key: <code>dig @ns.sslip.io my-key.k-v.io txt +short</code></li>
<li>To delete a key: <code>dig @ns.sslip.io delete.my-key.k-v.io txt +short</code></li>
</ul>
<h3>Notes:</h3>
<ul>
<li>Values are stored as TXT records.</li>
<li>Keys must be valid DNS subdomains, i.e. only alphanumerics and dashes, and cannot exceed 63 characters.
"my-key" is valid, but "my#key" is not.</li>
<li>Values can be one or more subdomains, i.e. alphanumerics, dashes, and <i>dots</i>, and are truncated to 63
characters. For example, <code>dig put.3.14159.pi.k-v.io txt +short</code> sets the the key "pi" to the value
"3.14159".</li>
<li>If you need special characters ("!@#$%^😊"), consider <a href=
"https://en.wikipedia.org/wiki/Base32">Base32</a> encoding, but remember that the padding character ("=") is
illegal.
</li>
<li>The three verbs are <code>put</code>, <code>get</code>, and <code>delete</code>, e.g. <code>dig
delete.pi.k-v.io txt +short</code>.</li>
<li>The verb <code>get</code> is the default verb: <code>dig pi.k-v.io txt +short</code> is identical to
<code>dig get.pi.k-v.io txt +short</code>.</li>
<li>There's no read security: when you set a key "my-super-secret-password" to the value "dont-tell-anyone",
anyone can read it.</li>
<li>There's no write security: you may set the key "best-rugby-team" to the value "all-blacks", and someone
else could change it a minute later to "springboks".</li>
<li>If you don't want someone to mess with your keys, you should probably use GUIDs or something fairly unique
as a key, e.g. <code>dig put.my-value.df616686-26e8-4da8-8104-a24aa0196bc7.k-v.io txt +short</code></li>
<li>I have no idea how well this system will scale.</li>
<li>I'll probably expire keys that haven't been accessed in a month, but I haven't written that code yet.</li>
<li>This project is very much in beta. Assume you may lose your keys at any time.</li>
</ul>
<h3>DNS Caching/Propagation:</h3>
<p>You don't need to specify our nameservers ("<code>@ns.sslip.io</code>") in your <code>dig</code> invocations
if you don't mind dealing with the vagaries of DNS caching and propagation. Here are some of the problems you'll
face:</p>
<ul>
<li>When you change or delete a key, it may take up to 3 minutes for the change to propagate due to DNS
caching.</li>
<li>DNS propagation/caching is particularly troublesome when doing multiple write operations on a key within a
3-minute period. The first one takes effect, the second one doesn't because it's cached at the upstream
nameservers & never reaches the <i>k-v.io</i> nameservers.</li>
</ul>
<h3>Technical Notes:</h3>
<ul>
<li>There are three servers that back this service: two in the USA, one in Singapore, on three different
platforms: AWS, Azure, and Google Cloud.</li>
<li>The underlying key-value store is an etcd cluster. I chose etcd for no particular reason other than it's
what Kubernetes uses.</li>
<li>The <i>k-v.io</i> source code is in Golang, is hosted at <a href=
"https://github.com/cunnie/sslip.io">GitHub</a>, and is licensed under the Apache 2.0 license.
</li>
<li>Much of the deployment information (terraform files, Kubernetes manifests) is also freely available at a
<a href="https://github.com/cunnie/deployments/tree/main/terraform/gcp/gke">different GitHub repo</a>, and some
of the workstation configuration is at a <a href=
"https://github.com/cunnie/bin/blob/main/install_ns-aws.sh">third GitHub repo</a>. Yes, it's an organizational
challenge.
</li>
<li>The DNS server code is a mash-up of key-value store and <a href="https://sslip.io">mapping hostnames with
embedded IP addresses to those addresses</a> (e.g. <code>127.0.0.1.sslip.io</code> → <code>127.0.0.1</code>).
In a perfect world, I'd have separated the codebase into two and deployed <i>k-v.io</i> on a second set of
three nameservers; however, I had neither the time nor the inclination to manage a 2nd set of DNS servers. So
here we are.
</li>
<li>If you have suggestions (or notice that something's broken), please open a <a href=
"https://github.com/cunnie/sslip.io/issues/new/choose">GitHub issue</a>.
</li>
<li>I was motivated to create this service because at my job we use S3 as a key-value store, and getting at the
keys is a challenge because we need the AWS credentials, and to get at those credentials we need to use CredHub
(a Vault-like secrets store). It's a lot of hoops to jump through to find out something as innocuous as the
version number of the latest candidate build.</li>
<li>
<a href="https://dnskv.com/">dnskv.com</a> is a similar service with a rich set of options.
</li>
</ul>
</div>
-->
</div>
</main><!-- /.container -->
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity=
"sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<script>
window.jQuery || document.write('<script src="/docs/4.3/assets/js/vendor/jquery-slim.min.js"><\/script>')
</script>
<script src="/docs/4.3/dist/js/bootstrap.bundle.min.js" integrity=
"sha384-xrRywqdh3PHs8keKZN+8zzc5TX0GRTLCcmivcbNJWm2rs5C8PRhcEn3czEjhAO9o" crossorigin="anonymous"></script>
</body>
</html>