Move along, nothing to read

<>Playing with the concepts around Cloud metadata services, specifically those that are network-based, rather than the ConfigDrive or serial port variants.

EC2 ensures that 169.254.169.254 is magically provide your instance with your data, and it won't be accessible to another instance. This is trivial to achieve if your instances are using routed or tap network; but is more complex if you are on a bridged network: the client will try to send the packets for 169.254.169.254 to the MAC of the default gateway.

So far I can force bridged packets that would otherwise be headed for the gateway to be routed locally (and put 169.254.169.254/32 locally on the host). I don't have a good way to associate the packets with a specific instance yet. Using kernel packet marks work, but isn't really scalable. Main requirement is that a simple web service should be able to uniquely identify the client, even if they try to spoof themselves (learn mac+IP of another instance on the same hypervisor & bridge, and ask for it's metadata from the wrong interface)

Variant 1

  ebtables -t nat -N metadata || ebtables -t nat -F metadata
for i in $(seq 0 20) ; do
   ebtables -t nat -A metadata -i vnet$i -j mark --mark-set $((256+$i)) --mark-target CONTINUE
done
ebtables -t broute -A metadata --limit 10/minute --limit-burst 2 --log --log-level debug --log-prefix "ebtables metadata" --log-ip
ebtables -t nat -A metadata -j redirect
ebtables -t nat -F PREROUTING
ebtables -t nat -A PREROUTING -p IPv4 --logical-in br0 --ip-src 169.254.169.254 -j metadata
ebtables -t nat -A PREROUTING -p IPv4 --logical-in br0 --ip-dst 169.254.169.254 -j metadata

Variant 2

ebtables -t broute -N metadata || ebtables -t broute -F metadata
ebtables -t broute -F BROUTING
ebtables -t broute -A BROUTING -p IPv4 --ip-src 169.254.169.254 -j metadata
ebtables -t broute -A BROUTING -p IPv4 --ip-dst 169.254.169.254 -j metadata
ebtables -t broute -A metadata --limit 10/minute --limit-burst 2 --log --log-level debug --log-prefix "ebtables metadata" --log-ip
# Repeat the marks if you want them.
ebtables -t broute -A metadata -j redirect