Where in the contract does it say you can't NAT IPv6?

Here's a quick and dirty way to multihome your house:

• connect to two different ISPs
• number your network in non-unique space (RFC1918 gives you a list!)
• setup NAT (PAT) to each ISP
• setup defaults to each, with different metrics (floating routes)
• if your first choice ISP is failing, go pull the plug facing them and the floating routes do the rest

But that only works for IPv4 right?

Well, no. It works just fine for IPv6 as well. It's just not standardized well...

 

I've been running my house for a while using this type of config that NATs both IPv4 and IPv6.  It isn't standards compliant, but every host can reach the IPv4 and the IPv6 internet.

I've also thrown a recursing name server into the mix so you can see hosts get their DNS setup via DHCP and via RA.

Have fun!

Here's my topology:

Here's a VyOS 1.4.3 config that should get you started if you want to try it out.

# The local LAN
# RFC-1918 for IPv4 and an IPv6 ULA /64
set interfaces ethernet eth1 address '10.11.12.1/24'
set interfaces ethernet eth1 address 'fd89:f9a4:7a42::1/64'
set interfaces ethernet eth1 description 'Guest LAN'

#
# Upstream with a local ISP with static addresses
set interfaces ethernet eth2 description 'Static IP WAN'
set interfaces ethernet eth2 address '2001:db8::2/126'
set interfaces ethernet eth2 address '192.0.2.138/29'

#
# upstream with Xfinity (dhcp, rather than SLAAC + DHCP-PD)
set interfaces ethernet eth3 description 'Xfinity WAN'
set interfaces ethernet eth3 address 'dhcp'
set interfaces ethernet eth3 address 'dhcpv6'
#
#  depref the IPv4 default route
set interfaces ethernet eth3 dhcp-options default-route-distance '200'


#
# setup IPv4 Masquerade NAT
set nat source rule 100 outbound-interface name 'eth3'
set nat source rule 100 source address '10.11.12.0/24'
set nat source rule 100 translation address 'masquerade'
set nat source rule 101 outbound-interface name 'eth2'
set nat source rule 101 source address '10.11.12.0/24'
set nat source rule 101 translation address 'masquerade'


#
# setup IPv6 Masquerade NAT
set nat66 source rule 100 outbound-interface name 'eth3'
set nat66 source rule 100 source prefix 'fd89:f9a4:7a42::/64'
set nat66 source rule 100 translation address 'masquerade'
set nat66 source rule 101 outbound-interface name 'eth2'
set nat66 source rule 101 source prefix 'fd89:f9a4:7a42::/64'
set nat66 source rule 101 translation address 'masquerade'


#
# Ensure the IPv4 default route has a prefernce above that
# on the DHCP derived route
set protocols static route 0.0.0.0/0 next-hop 192.0.2.137 distance '100'


#
# Set default route preferences in IPv6
set protocols static route6 ::/0 interface eth3 distance '200'
set protocols static route6 ::/0 next-hop 2001:db8::1 distance '100'


#
# setup caching DNS
set service dns forwarding allow-from '10.11.12.0/24'
set service dns forwarding allow-from 'fd89:f9a4:7a42::/48'
set service dns forwarding allow-from '::1/128'
set service dns forwarding allow-from '127.0.0.1/32'

set service dns forwarding listen-address '110.11.12.1'
set service dns forwarding listen-address 'fd89:f9a4:7a42::1'
set service dns forwarding listen-address '::1'
set service dns forwarding listen-address '127.0.0.1'

#
# Make sure SLAAC works
set service router-advert interface eth1 dnssl 'arpa.home'
set service router-advert interface eth1 name-server 'fd89:f9a4:7a42::1'
set service router-advert interface eth1 prefix fd89:f9a4:7a42::/64

#
# setup a DHCP service for IPv4
set service dhcp-server listen-address '10.11.12.1'
set service dhcp-server shared-network-name Guest-IPv4 authoritative
set service dhcp-server shared-network-name Guest-IPv4 name-server '10.11.12.1'
set service dhcp-server shared-network-name Guest-IPv4 subnet 10.11.12.0/24 default-router '10.11.12.1'
set service dhcp-server shared-network-name Guest-IPv4 subnet 10.11.12.0/24 domain-search 'home.arpa'
set service dhcp-server shared-network-name Guest-IPv4 subnet 10.11.12.0/24 lease '3600'
set service dhcp-server shared-network-name Guest-IPv4 subnet 10.11.12.0/24 range 0 start '10.11.12.101'
set service dhcp-server shared-network-name Guest-IPv4 subnet 10.11.12.0/24 range 0 stop '10.11.12.200'

#
# setup home.arpa so it has at least one host
set system static-host-mapping host-name router.home.arpa inet '10.11.12.1'
set system static-host-mapping host-name router.home.arpa inet 'fd89:f9a4:7a42::1'

There's a new Sheriff in Town -- How a sprinkler system helped me recognize the future

Ever since arduinos and other cheap microcontrollers have been available, I've thought about building a sprinkler system.  On a raspberry Pi it would be so simple:

• build a schedule composed of CSVs: "day of week", "start time", and "stop time" as the only values
• build a script to start/stop a zone by turning relays on/off
• drive the whole thing via cron after parsing the CSV

The piece that kept me back was the schedule builder.  Simple schedules would be easy, but I kept finding other projects that would be rather complex.  What I really wanted was a week long calendar view with UI elements I could drag around and resize.

I've done enough programming to know this was a simple, but intricate job.  I could likely built it with some existing libraries, and a very small amount of logic.  Heck, it's been innumerable times in the past by lots of people.  I could likely find a implementation somewhere if I dedicated some time to it.

Since the existing system performed adequately, the time required to research, build, and test any such system simply wasn't worth it.

Today the idea hit me again, and I decided that I'd let some AI engines help me.  I gave three different readily available LLMs the following:

I want an app that runs completely within a web browser as a single HTML file. The goal is to use a graphic interface to produce a schedule for turning a sprinkler system on/off.

It should allow the graphical construction of a schedule, much like any calendaring program. Mimic google calendaring for UI ideas.

The display should show a week of days horizontally, and a gird of hours midight to midnight from top to bottom.

Blocks should be moveable -- both horizontally and vertically with a drag mouse movement

There should be a button that converts the current week into CSV with each line showing: day of week, time, action (start/stop)

The ability to import an existing CSV should exist too

Every one of them gave me a usable GUI within 3 minutes.  Click, drag, edit times, it all works.

We are in a very different world.

So, you want to try IPv6?

I decided I should build an access router using VyOS 1.4.3. I thought you might like the journey.

I drew a lot of useful config snippets and inspiration from https://blog.daknob.net/ipv6-first-with-vyos/. Make sure you take a look at their posts.

I recently brought an Xfinity Now! connection into my house. It runs a bit over 100M/20M, so qualifies as broadband according to the lastest definition from the FCC here in the USA. At $30/month, it's pretty cheap.

My primary connection is with a WISP that I work for on/off, and we use the link to test new configurations, etc. So this Xfinity link is a backup for the times when I break the primary connection.

Knowing that Xfinity does a good job with IPv6, I thought I'd build up soemthing to show:

• a VyOS config so people can implement it themselves if they want
• the use of a management VRF
• integrated recursive DNS (you don't need external resolvers who sell your data)
• dual-stack IPv4 and IPv6 capabilities
• IPv6 only with some IPv4 transition
• use of IPv6 ULA for local services (DNS in this case)

Here's a diagram of the network that built up:

You'll note four interfaces in use:

• eth0 -- a VRF used only for managment
• eth1 -- our WAN connection to Xfinity
• eth2 -- a dual-stack LAN
• eth3 -- an IPv6-only LAN with DNS64/NAT64

I'm likely to add more LANs to this configuration as I go. In particular, I want a reference network using NAT66, but for now we'll live with the examples above.

First, I built up a management interface (eth0) that sits in its own VRF, then built up what you'd expect for an IPv4 environment.

#
# Build the VRF
set vrf name Mgmt table '101'
#
#setup the interface, and put it into the VRF
set interfaces ethernet eth0 address '198.51.100.11/24'
set interfaces ethernet eth0 description 'Management'
set interfaces ethernet eth0 vrf 'Mgmt'
#
# Make sure the VRF has a default route
set vrf name Mgmt protocols static route 0.0.0.0/0 next-hop 198.51.100.1 distance '200'
#
# make ssh works inside the VRF rather than in the data plane of the router
set service ssh vrf 'Mgmt'
#
# setup the WAN link
set interfaces ethernet eth1 description 'Xfinity WAN'
set interfaces ethernet eth1 address 'dhcp'
#
# setup the LAN link
set interfaces ethernet eth2 description 'LAN'
set interfaces ethernet eth2 address '192.0.2.1/24'
#
# Setup a DHCP pool for the LAN
set service dhcp-server listen-address '192.0.2.1'
set service dhcp-server shared-network-name Xfinity-Dual-Stack authoritative
set service dhcp-server shared-network-name Xfinity-Dual-Stack subnet 192.0.2.0/24 default-router '192.0.2.1'
set service dhcp-server shared-network-name Xfinity-Dual-Stack subnet 192.0.2.0/24 lease '3600'
set service dhcp-server shared-network-name Xfinity-Dual-Stack subnet 192.0.2.0/24 range 0 start '192.0.2.101'
set service dhcp-server shared-network-name Xfinity-Dual-Stack subnet 192.0.2.0/24 range 0 stop '192.0.2.200'
#
# setup NAT masquarade for the LAN
set nat source rule 100 outbound-interface name 'eth1'
set nat source rule 100 source address '192.0.2.0/24'
set nat source rule 100 translation address 'masquerade'
#
# Setup a recursive DNS server, and let it cache 10000 names
set service dns forwarding allow-from '192.0.2.0/24'
set service dns forwarding dnssec 'validate'
set service dns forwarding listen-address '127.0.0.1'
set service dns forwarding listen-address '192.0.2.1'
set service dns forwarding cache-size '10000'
#
# Make sure the dhcp-server hands out this information
set service dhcp-server shared-network-name Xfinity-Dual-Stack name-server '192.0.2.1'

Cool! Now we have IPv4 connecvitiy. Most people stop here!

Why stop with leagcy protocols? Xfinity will give us the ability to run IPv6 too!

It don't cost nuthin'

#
# First, we get an address from the provider via SLAAC.
set interfaces ethernet eth1 ipv6 address autoconf
#
# Next, we request a /56 as PD #1  (Xifiniy will only give us a /60,
# but I can dream, right?)
set interfaces ethernet eth1 dhcpv6-options pd 1 length '56'
#
# Now, we tell VyOS to build up the IPv6 address for eth2 based on the
# IA-PD we received on eth1.  It gets built up in the form:
#        [prefix]:[sla-id]:[address]
set interfaces ethernet eth1 dhcpv6-options pd 1 interface eth2 sla-id '0'
set interfaces ethernet eth1 dhcpv6-options pd 1 interface eth2 address '1'

Now the router has a dynamic address based on what our provider sends us.

Our LAN clients can build their own network addresses via SLAAC if we start router advertisements on eth2:

#
# advertise all the prefixes on the interface
set service router-advert interface eth2 prefix ::/64

At this point, anyone on eth2 has dual-stack capability, but only the IPv4 folks have DNS resolution. We have a DNS resolver running on the VyOS host, but for IPv4, it is using non-unique addresses for local communication. This local addressing is NATted before leaving the local network.

(I'm publishing this using the addresses assigned for test/examples as defined in RFC5735. You might want to substitue addresses from RFC1918 if you follow this config.)

Our IPv6 setup has dynamically assigned unique addressing. We don't have any means to setup the DNS server to listen on those addresses, becasue we don't know them!

To address this situation, we're going to utilize IPv6 ULA addresses. This block of space is defined in RFC4193, and has similar use to the IPv4 addresses defined in RFC1918.

IPv6 has an example network (2001:db8::/32), but there is not clear ULA space set aside for examples. In this config, I'm going to use fd89:73ab:e943::/48. You should generate your own prefix based on RFC4193.

At the time of this writing, there is a site that will generate a prefix for you: https://www.unique-local-ipv6.com/

Another important note at this point. We are going to have both GUA and ULA addresses on the LAN hosts. The use of mutlitple addresses on an IPv6 host is expected, but sometimes feels a bit strange to folks habituated to IPv4.

We're using fd89:73ab:e943::/48 for our ULA block, but this LAN will be using fd89:73ab:e943:2::/64.

#
# anchor the router in the ULA space, by giving it a loopback address in that space
set interfaces loopback lo address 'fd89:73ab:e943::1/128'
#
#  Assign a static ULA address to the interface (and point out we're dual-stack!)
set interfaces ethernet eth2 address 'fd89:73ab:e943:2::1/64'
set interfaces ethernet eth2 description 'dual-stack LAN'
#
# Tell the LAN hosts that they should generate SLACC addresses with this ULA block
#    (this isn't actually neeed, because we did it earlier with ::/64, but
#     leaving things this explicit might help someone trying to reason this out)
set service router-advert interface eth2 prefix fd89:73ab:e943:2::/64
#
# Start advertising the DNS server via RA
set service router-advert interface eth2 name-server 'fd89:73ab:e943:2::1'

At this point, we have a LAN that has dual-stack capabilities!

IPv4 address and DNS server information are delivered via DHCP. IPv6 hosts will hear RAs instructing them to generate their owan addgesses (both GUA and ULA). Those RAs will also indicate the DNS server address as well.

In IPv4, a side effect of NAT denies inbound connections. We can deny inbound IPv6 connections in a similar fashion with a stateful firewall. One example follows:

#
# build a stateful firewall so inbound connections are denied
set firewall group interface-group LAN interface 'eth2'
set firewall group interface-group WAN interface 'eth1'
set firewall ipv6 forward filter default-action 'drop'
set firewall ipv6 forward filter rule 10 action 'accept'
set firewall ipv6 forward filter rule 10 state 'established'
set firewall ipv6 forward filter rule 10 state 'related'
set firewall ipv6 forward filter rule 20 action 'accept'
set firewall ipv6 forward filter rule 20 protocol 'ipv6-icmp'
set firewall ipv6 forward filter rule 30 action 'accept'
set firewall ipv6 forward filter rule 30 inbound-interface group 'LAN'
set firewall ipv6 forward filter rule 90 action 'drop'
set firewall ipv6 forward filter rule 90 inbound-interface group 'WAN'

Now, we'll setup the final LAN as an IPv6 only network:

#
# setup our static ULA assignement for eth3
set interfaces ethernet eth3 description 'IPv6 only - NAT64'
set interfaces ethernet eth3 address 'fd89:73ab:e943:3::1/64'
#
# add our dynamic allocation as well for GUA
set interfaces ethernet eth1 dhcpv6-options pd 1 interface eth3 address '1'
set interfaces ethernet eth1 dhcpv6-options pd 1 interface eth3 sla-id '1'
#
# Setup RA for this LAN (we'll trust the "prefix ::/64" to advertise both GUA and ULA
set service router-advert interface eth3 prefix ::/64
set service router-advert interface eth3 name-server 'fd89:73ab:e943:3::1'
#
# Tell the DNS server that it's ok to listen to the new LAN
set service dns forwarding listen-address 'fd89:73ab:e943:3::1'

We now have a working IPv6-only LAN!

With a little bit more of configuration, we can take advantage of a pair of IPv6 transtition technologies referred to as DNS64/NAT64. I'll leave deeper study to the casual reader, but a quick overview of this is:

• DNS64 -- In cases of host that have A records, but no AAAA records, we'll synthesize a AAAA that uses a special IPv6 prefix
• NAT64 -- Whenever the special prefix is used, we'll NAT from IPv6 to IPv4 to proxy the service requested

In this case, we're going to use the default IPv6 DNS64 prefix: 64:ff9b::/96

Let's say you want to connect to example.com which has an A record for 192.0.2.123. The DNS transaction will look like:

• you request an AAAA record from the DNS server (you're IPv6 only, remember?)
• the DNS server will note that no AAAA is published, but there is an A record
• the DNS servie will create an AAAA record based on the A record, and return it to you

In essence, the DNS serve will LIE to you. The AAAA record in this case would be: 64:ff9b::192.0.2.123

When you iniate a connection to anything within the 64:ff9b::/96 prefix, the router will undertand that it needs to proxy an IPv4 connection to the appropriate host.

Although not strictly needed, RFC8781 also points out good reasons to let the LAN hosts know the NAT64 prefix.

The VyOS config is rather short for all of this:

#
# tell the DNS server we're going to do DNS64
set service dns forwarding dns64-prefix '64:ff9b::/96'
#
# tell the router to do NAT64
set nat64 source rule 1 source prefix '64:ff9b::/96'
#
# use RAs to tell the LAN hosts the NAT64 prefix
set service router-advert interface eth3 nat64prefix 64:ff9b::/96
#
# add this LAN to the stateful firewall setup
set firewall group interface-group LAN interface 'eth3'

At this point, there are two working LANs:

1. eth2 -- dual stack IPv4/6
2. eth3 -- IPv6 only with DNS64/NAT64

I'll probably add a few more LANs before I'm done to document other IPv6 transitional technologies. For now, I'm pretty happy with this setup.

As I started to publish this, I realized that the router itself didn't have DNS services! I decided that it should use it's own recursion and use IPv6 for all queries, so add this:

#
# allow queries from the loopback
set service dns forwarding listen-address '::1'
#
# send queries to the loopback
set system name-server '::1'

~

Where are the rants?

I need to apologize.  I started this as a place to rant.  It hasn't turned out that way.

At first, I wasn't happy about the state of our government, and I wanted to proclaim ideas that I thought would help.  I didn't think they'd get anywhere, but I wanted to make sure they were published.  I'm still not happy about all aspects of our government, but at least I can point to some of my early posts and say, "See?  I tried to give you some options!"

Since then, I've published things that stuck me as interesting or fun.

But I think I need to rant more...

Edge Computing (or something)

 A little over a month ago (October 4/5, 2023 if you're reading this in the far future), I was invited to be a delegate at Gestalt IT's second Edge Field Day (#EFD2).  I'd participated in a couple of earlier Field Days, so I knew the format:   Vendors would present information to a group of technical folks and discussion would ensue.

At the moment, the core of my consulting work lies in helping a WISP with networking issues, and giving technical advice to the State of New Mexico's Office of Broadband Expansion and Access.  Also, I run the ABQIX, an Internet Exchange in multiple data centers in Albuquerque, New Mexico.

I don't really design/implement things that I think of as "Edge Computing," so I wasn't really sure what value I would add.  But I knew I'd learn a lot, and am old enough that I'm willing to ask questions that others might feel are obvious without getting embarrassed.  Also, I figured I'd be able to add comic relief if nothing else to the fray.

On site, we had a roundtable discussion with all of the delegates to talk about Edge Computing.  In the end, we talked about a number of issues important to IT workloads, security, and many other points.  But I was still fuzzy on Edge Computing.

Next up was a presentation from Solidigm.  Their presentation showcased their SSD products.  They've got everything:  speed, reliability, physical form, rugged packaging, storage density.  At the end of their presentation, I was convinced that if I was an OEM, I'd be considering the use of their gear.  But I'm not sure I knew a lot more about Edge Computing.

We then heard from StorMagic.  They showed us their SvSAN product which was quite cool.  Running on top of hypervisors, you can scale the underlying hardware to create the performance level you need.  They also have some cool tools for managing these storage devices as you scale -- in fact, they let us see their Edge Control tool before releasing info publically.  I was starting to wonder if Edge Computing was really about storage...

Edge Field Day #2 ended with an introduction to NodeWeaver.  I know I didn't grok this product, but it spoke to me.  A slim hypervisor with a slick install system that lets you build (yet another) hyperconverged platform.  A compute and storage swiss-army knife that runs on virtually any x86 hardware, so you can mix/match to meet your performance needs -- just want you need for Edge Computing.  Right?

I went home from Edge Field Day with a heavy heart.  I seriously wondering if I'd been worth anyone's time.  We had good talks on technology with great people, but I still wasn't sure what Edge Computing was.

On October 20, David Linthicum published an article titled "Whatever Happened to Edge Computing?"  The title made me wonder if I could stop worrying about Edge Computing because I'd already missed the boat.  (Actually my take on David's thoughts indicate that Edge Computing is alive and well -- but the wealth of possible applications make it difficult to define meaningful standards.)

After the last few weeks, I've decided that I'll define Edge Computing for myself until someone can get me a better definition.  To wit:

Edge Computing is a optimization strategy that involves distributed computing.

Or something.

At least it isn't the typical consultant weasel words: "It depends."


Postscript:

Thanks go out to Gestalt IT, Solidigm, StorMagic, and NodeWeaver to making things rattle around my head for a couple of weeks.  I didn't realize how much this was bothering me until this article started to flow.  My conclusion is overly broad -- but so the the term I tried to define.

David Linthicum gets credit for writing an article that helped me gel my thoughts.

An Old Sysadmin Tries Out ChatGPT

(Editorial note: Pretend this post is a Star Trek episode with folks spouting techno-babble, use the words as place holders, and feel the flow of the story...)

A while back, I built some anycasting DNS resolvers using bind on a loopback, and FRR talking OSPF to a router.  I finally decided I should hammer together a script to turn off FRR if there was something wrong with the bind process so the servers would pull themselves out of service.

Why not try this new ChatGPT thingee?

I won't bore you with the code that was created, but you'll see my thoughts, and the input I gave to ChatGPT as I worked through this problem until... the end?

---

The Ramp Room

If you've ever spent time doing telecommunication work in Albuquerque, NM, the odds are you've had to work at 505 Marquette NW.  If you're local, you just talk about "505" and everyone knows what you mean.

505 was built during 1965 and 1966, and isn't a bad looking place, but it is showing it's age. Wikipedia quotes a July 10, 1966 Albuquerque Journal article: "It has twelve floors of office space above a wider six-story base which incorporates a parking garage on floors 2–6."  The Wikipedia article has this  picture:

Now that Albuquerque has become a center for shooting TV and Movie productions, you will sometimes see this building in the background.  Perhaps my favorite is from the Netflix series Daybreak where the building gets an FX makeover to fit the apocalyptic theme of the show:

I can't find a picture on the interwebs, so you have to settle for the one I took while watching Daybreak on Netflix.  Note: Palm trees, background hills, and building damage are all simulated.  I do, however, get my home Internet connectivity from the antenna on the roof.

Did you note that the parking garage was on floors two through six?  That means you have to drive up a ramp to get to the parking levels.  This architectural need has lead to one of  Albuqueruque's telecom black holes known as the Ramp Room.

To enter the Ramp Room, you need to get the the garage ramp, where air conditioning condensers congregate to release the heat of the inner building.  In between the first and second floors you'll find a non-descript door:

Upon opening the door, you'll find that it immediately blocks your way.  It feels like you aren't really welcome, but you move forward anyway...

...And end up in a narrow hallway.

Behind the door at the end of the hall way you immediately see a rack of batteries that were put in twenty years ago and never maintained. 

One step into the room and you can see the building punch downs -- the metallic path over which the business conversations from fifty years ago used to pulse.  Yet there is also a veneer of modern fiber optic communications that reside here as well.  In fact, they are the mainstay of the Ramp Room.

In every city, there is a nexus of communication that isn't ensconced in a pristine data center.  A place where various players gather.  But not because they really want to do so.  Rather, their customers demand they work together.

The path of least resistance nets small, out-of-the-way rooms where fiber optics from multiple carriers co-exist.  A place that isn't official, but becomes important.

Some players have been around for a long time.   They've been sold, bought others, been forced to divest assets, and any number of other acquisition issues.  Sometimes you forget what they're called now, and just refer to them by their "old" names.  After a while it you stop worrying about what to call them.

They have, and will persist.

Other are younger.  More nimble.  Willing to do things out of the ordinary, and force the old guard to play their way.

For a time.

They will become the old guard for a new generation.

Years roll on, and old gear is left in rooms like this.  Forgotten.  Useless technology from the past interwoven with the modern.

Old gear isn't ever removed.  It sits and drinks power from the building.

No one owns it, so no one cleans it.

Everyone is afraid to shut anything off.  Few know how to tell if something is still in use or not.

Within this building are a multitude of other places to interconnect -- but none have the diversity of the Ramp Room.

ModemCast -- a podcast

I tend to babble a lot.  Sometimes I find people that babble about some of the same things.  My friend Nick has talked me into helping him babble as well.  His take on things is here:
  https://forwardingplane.net/2021/01/21/modemcast-podcast/

You might even enjoy some of our mutual babbling:
    https://www.modem.show/

If you find it to your liking, do not blame me.

Joe's Internet -- an IGP adventure

Before we can setup inter-network links, Joe's Internet (J-Inet)  need to get the internal network in order.  We'll implement an Internal Gateway Protocol (IGP) so all of our infrastructure will be able communicate with other parts of our network.

The IGP's job is to advertise what it knows to all of it's neighbors.  If there are multiple paths to a destination, it will pick the best of those available, and pick alternates in case of a path failure.  The IGP ensures communication resiliency as long as there are redundant paths.

Quick network overview

We're going to make sure each router has a single IP address on a "loopback" interface.  Loopbacks give you a management interface on the router that is not dependant upon any given interconnection link's state.  As long as the IGP is running properly, every router will know how to get to every other router via their loopback regardless of what path is in use.

Our infrastructure addressing is going to include the networks that interconnect the routers, and the loopback addresses on each router:

J-Inet's IGP Map.  Click on it for more detail.

IGP by psuedo-code

No matter the platform, we have a few things we need to do to ensure our goal.  Those steps come down to:

1. setup loopback interface address
2. set global IGP parameters
3. get IGP to adopt the loopback
4. for each infrastructure link (one or more)
1. setup interface address
2. get IGP to adopt the interface

Pretty simple, really.  We're going to use OSPF in this example.  I'll assume a passing familiarity with OSPF, and just point out that we will only be using Area 0.  There's no reason to do things in a fashion more complicated than a single area.  (If that doesn't make sense, find some background material on OSPF, and come back.)

Also, we're going to pretend these routers run something that looks a lot like Cisco IOS from a syntax standpoint. Take a look at a config for routers R1 and R2:

!  Router R1
!
! global IGP parameters
router ospf 1
 router-id 192.168.92.1
! setup loopback, and add to IGP
interface Loopback20
 ip address 192.168.92.1 255.255.255.255
 ip ospf 1 area 0
! setup infrastructure link, and add to IGP
interface ether 1
  description Link to R2
  ip address 192.168.92.129 255.255.255.252
  ip ospf 1 area 0
! setup infrastructure link, and add to IGP
interface ether 2
  description Link to R3
  ip address 192.168.92.137 255.255.255.252
  ip ospf 1 area 0

! Router R2
!
! global IGP parameters
router ospf 1
 router-id 192.168.92.2
! setup loopback, and add to IGP
interface Loopback20
 ip address 192.168.92.2 255.255.255.255
 ip ospf 1 area 0
! setup infrastructure link, and add to IGP
interface ether 1
  description Link to R1
  ip address 192.168.92.130 255.255.255.252
  ip ospf 1 area 0
! setup infrastructure link, and add to IGP
interface ether 2
  description Link to R3
  ip address 192.168.92.149 255.255.255.252
  ip ospf 1 area 0
! setup infrastructure link, and add to IGP
interface ether 3
  description Link to R3
  ip address 192.168.92.145 255.255.255.252
  ip ospf 1 area 0

I will leave the construction of configs for routers R3 and R4 to our readers.

While your setting things up on a Cisco router, you will find the following commands useful to help you track your progress:

• show ip ospf neighbor
• show ip route
• show ip route ospf

Adding Complications to the IGP setup

Most of the time, I  tell people to stick to the simplest setup they can.  The configs above should let you build a simple IGP to keep your network running.  However, there are some things you may encounter that require a bit more thought into your setup and will force your to consult your platform's documentation.  A couple of the more common issues and a method to use them are:

• password / encryption / message-digest for IGP on links
• link type (point-to-point vs. point-to-multipoint, etc.)
• changing link metrics

Consult your platform documentation, but to implement the above features, we could have added something like the following to our configs:

router ospf 1
   area 0 authentication message-digest
   auto-cost reference-bandwidth 10000
!
interface ether-whatever
   ip ospf network point-to-point
   ip ospf message-digest-key 1 md5 <some key/password>

Conclusion

All of our routers now know how to reach each other because the IGP keeps them all informed.  If we add new routers, we simply need to add them into the IGP, and they will be added to the reachability information that all of the routers share.

Our eventual goal is to ensure that the customers share this type of reachability.  To do that, we will use BGP to leverage the IGP information for anything else we need to reach on this network.  That will be the subject of the next episode of J-Inet...

Joe's Internet -- Addressing and Interconnection

In the last installment, Routing Policy -- interconnecting with other networks, we talked about the business relationships Joe's Internet (J-Inet) might have with other networks.  Now we'll look a little closer at the technical side of how those relationships are enacted on the network itself.

Before we get too much closer to J-Inet, I want to point out that I'm going to be using a single network model for continuity between posts.  I've worked out IP addressing (v4 & v6), consistent routing policies, and a topology that should suffice for many posts. I've got several posts lined up that should help show at least one path to building a scalable network.

The Interconnects

J-Inet has two transit providers (Transit 1 & 2), three customers (Customer A, B, & C), and a single Peer (Peer 1).  Customer C is a bit of stretch for this conversation.  They aren't an autonomous system, but we'll look at them anyway, as it may lead some readers to revelations they might not have otherwise.

J-Inet has four Points of Presence (POPs) where equipment is housed.  Most customers of J-Inet are like Customer C.  They simply buy transit from J-Inet.  These customers actually make up the bulk of J-Inet's revenue.  But they're pretty boring from an network perspective.

The Network, Prefixes, and Policy

The network itself is pretty simple, and looks like this:

Over view J-Inet's network and interconnects.  Click on it for more detail

J-Inet has two blocks of address space it uses internally, that we will advertise to anyone that interconnects.  Those prefixes are: 172.16.32.0/21 and 192.168.92.0/22

Customer A uses 172.16.36.0/24.  That block comes from J-Inet's prefixes, but they are allowed to use it as long as they are a customer of J-Inet.  J-Inet also allows Customer A to advertise this prefix across any interconnects they may have with other networks.  J-Inet will pass all of the prefixes it knows about to Customer A.

Customer B uses 10.193.32.0/22, which was allocated to them from their Regional Internet Registry.  They advertise this prefix to us, and any other interconnects they have according to their own routing policy. J-Inet will pass advertise all routes to Customer B.

Customer C uses 192.168.93.112/28.  They don't have any other interconnects, so they aren't really an autonomous system (AS).  They're in this mix as a kind of anti-example -- a customer that really only acts as an extension of J-Inet's network and routing policy.  Neither Customer C nor J-Inet will advertise prefixes to one another.

Peer 1 is another ISP that happens to have equipment in the same building that houses POP 1.  J-Inet and the Peer 1 decided that they should interconnect directly.  They will be announcing 172.16.34.0/24 to J-Inet.  J-Inet will pass internal and customer prefixes to Peer 1, but will not send any routes learned from transit providers.

Transit 1 and Transit 2 are networks that have customers and other interconnects.  Since they are transit providers, they have promised to get traffic to any destination. However, they bouth announce prefixes to us, J-Inet can make an informed decision about which network should be used for any given prefix.  J-Inet will announce internal routes and customer routes to both transit providers.  J-Inet will not advertise routes learned from Peer1.  Lastly, J-Inet will not advertise any transit routes back to either transit provider.  (i.e. J-Inet will not allow transit to the these providers.)

At this point, we have a pretty good idea of J-Inet's network topology and how it interconnects with customers, peers and transit providers.  In our next installment, we'll look at some of the internal housekeeping of J-Inet's network is put together, and later use this internal configuration to make our interconnects simpler.

Routing Policy -- interconnecting with other networks

The next few posts about Joe's Internet (J-Inet) will start with our routing policy, and refine it based on some ideas concerning how to describe the interconnections between networks.  We'll get a look at J-Inet's network soon, and eventually we'll see how the router configurations enact the routing policy.  For now, let's get a grounding in how we can describe links between networks, and how they relate with routing policy.

J-Inet has a simple management philosophy when it comes to interconnecting with other networks.  In summary:

• keep things as simple as possible
• minimize costs
• maximize revenue

These thoughts are going to be the basis of our routing policy -- make money while keeping things simple and cheap.

Types of interconnection

The types of networks we are discussing are referred to as Autonomous Systems (ASes).  Within any AS are destinations referred to as prefixes or routes.  When an AS interconnects with another, they advertise their prefixes to the other -- essentially offering a path to that destination.

It turns out that if you learn about three simple types of business relationships, you can understand the vast majority of AS interconnection types:

• Transit  -- If you buy Internet connectivity from someone, they are a transit provider.  You can hand them traffic, and they will deliver it to the desired recipient, or will pass that traffic across their network to a provider that can get the traffic closer to its destination.  For our purposes, if you pay people to take your traffic, you're buying transit.
• Customer -- If you supply transit to another network, they are a customer.  The simplest customer relies on you completely, and more complex customers look at you in the same way you look at your transit provider(s).  For our purposes, customers pay you to be their transit provider.
• Peering -- Sometimes, networks find that it is mutually advantageous to interconnect, and they decide to do so without either side paying for the privilege.  Common customers, shared geographic regions, and other factors may influence if networks will become peers.  We aren't going to worry about *why* some of our neighbors a peers -- we will just know that they are.

Egress Policy

As the number of interconnections increase, the odds of having more than a single path to a given prefix grows.  Our policy will give us guidance about which type of interconnection we wish to use, and we will rank prefix advertisements in this order:

1. Customer -- Goal:  Maximize revenue.  Always use this path if available.
2. Peers -- Goal: Minimize cost: Use this path before using one to a transit provider.
3. Transit -- Goal: Minimize cost.  Use this path only if no better path exists.

Prefix Advertisement Policy

Business policy shaped how we prioritize the choice of multiple egress paths.  Likewise, what we choose to advertise across an interconnection point is shaped by our policy as well.  In this case the reasoning becomes:

• Transit  -- Goal: Minimize costs.  Only advertise internal prefixes and those that your are paid to carry (e.g. prefixes learned from Customers), thus minimizing traffic.
• Customer -- Goal: Maximize revenue.  Advertise every prefix you know about to maximize traffic.
• Peering -- Goal: Minimize costs.  Advertise the same set of prefixes as sent to Transit providers, in the hope of reducing transit costs.

Parting thoughts and a glimpse of the future

By describing AS interconnections as one of three different types, we've been able to express some routing goals that will support our business goals.  In the next installment, we'll look at a map of J-Inet, start to see actual prefixes, and classify the inter-AS interconnects.

In other future installments, we will start to put together router configurations that will allow us to enact this routing policy.  At first, we'll use Cisco vocabulary, and stick with IPv4, but the plan is to use a single network to show the use of other vendor's gear as well. Stick around, this should be fun...

Project Hummingbird

Originally posted to Facebook on January 12, 2018 I thought this should be available outside of Facebook:

This may be long, but I came upon an old floppy (a floppy!) today, and found the drawing you see here. I'm about to turn 50, so let me spin a tale from when I wasn't even half my current age...

Project Hummingbird

Some friends and I had to build "something" for our engineering senior design course. A friend had a model airplane remote control setup and wanted to play with "ducted fans." We came up with an idea for an aircraft we had a hard time describing to people.

Rather than wings, we were going to generate lift with pure thrust from the ducted fans! It was going to be the prototype for vertical take off and landing craft, and be a darn cool model to fly around.

Today, you can go buy a drone, and fly something similar, but in 1990, this was something no one had seen in radio controlled aircraft.

We had *NO* idea how much we'd bitten off.

To fit our budget constraint, we could only afford two of the ducted fan assemblies and engines. So we had to design a set of ducts to split the two sets of thrust into four so that we could control the craft.

To generate the thrust required, we had to have ducts that were over five inches in diameter. To keep control, we needed the nozzles over 20 inches from the center of the craft. This means that the craft needed to be about a foot high, and nearly four feet across.

Naturally, we designed it as a saucer.

We managed to fabricate one set of ducts, and showed that it generated more thrust than we had calculated. We got prototype control circuits mocked up before we realized we couldn't afford the gyroscopes available at the time. We learned the types of things you're supposed to learn in a senior design course.

But I *still* want to see that thing fly...

Joe's Internet -- a place so see how layer 3 works

I've learned a lot about networking from others over the years.  Usenet news, conferences, mailing lists, forums, blogs, and podcasts have been central to my learning at one point or another.  I was always learning from others, so I never really felt that I had all that much to contribute.

Lately, I've realized that a different voice telling the same story might help someone, so I decided I should start giving something to the community.

Over the last few years, I played around with the idea of Joe's Internet (J-Inet).  I've already posted one story about J-Inet and a foray into IPv6.  (In fact, the response to that addressing adventure was a big prompt to get me to write more.)

In the next few posts I'll show you the architecture of J-Inet.  We'll talk about routing policy, or the plain language that describes how we route Internet traffic.  As things progress, we'll get into router
configurations required to enact that routing policy.

I'll start with generic Cisco IOS configurations, because that's what I learned starting back in the 90s.  However, I'll swing back around and show how it can be done with different vendor's gear.  In the end, I hope people will figure out that all gear should play nicely together and that heterogeneous networking can work well.

Keep watching this space.  If I get my way, you'll learn something -- or at least get a new perspective of things you already know.  I'd like to hear your perspectives as well.

Welcome to J-Inet.  We push packets.

The Brothers WISP - Five (or six) tips.

Greg Sowell, of the Brothers WISP, talked to me about networking tips!  Greg and the Brothers do a great job of keeping a community of like-minded folks talking and helping each other.

If you're interested in watching things, take a look:

http://thebrotherswisp.com/index.php/thebrotherswisp-top-5-tips-john-osmon/

Joe's Internet goes IPv6 -- an adventure in numbering plans

This is a sample IPv6 number plan that was put together with good intentions.  It will not be complete, and it isn't likely to be the answer you're looking for -- but it may help you find your way to developing one of your own.  Feel free to use any of the ideas here.

I'm assuming that people taking the time to read this will also be familiar with acronyms and jargon used.  If you run into a term you don't know, ask about it in the comments.

 Joe's Internet ventures into the IPv6 World

Joe's Internet is a WISP that has just  been given the right to use a block of IPv6 address from their Regional Internet Registry (RIR).  Currently, Joe's Internet has 37 towers/POPs that they use to service customers, but have just won a bid to deploy 13 more POPs.  That means 50 POPs that require numbering.

This numbering plan will give Joe's Internet the ability to give every customer unique address blocks for their Internet use.  There will be plenty of addresses for customer use, and no need to force multiple clients to proxy behind a single address.

Some generic assumptions about this numbering plan include:

• IPv6 block is:  2001:db8::/32
• All networks will be assigned a /64 prefix (industry standard)
• All customers will be assigned a /48 (giving them 65k networks to use as they wish)
• Each POP will be assigned a /40

POP Codes

Let's assign each POP a 2 digit hex code -- that gives us room for 256 POPs.  We haven't used a quarter of our address space with this idea -- so have plenty of room to grow, or to make changes down the line if the original plan is too wasteful.

We'll have two example POPs to use for this documentation:
   1A -- Springfield
   1B -- Winterfield

Anatomy of an IPv6 address at Joe's Internet

With the above assumptions, we can have an address structure as below:


    --------------------------------   Joe's /32
    |         ----------------------   POP code
    |         |   ------------------   Customer designation
    |         |   |     ------------   Customer's 2^16 LANs
    |         |   |     |      -----   Local LAN
    |         |   |     |      |
|---------| |--||--| |----|  |-------|
[2001:0DB8]:[XX][YY]:[AAAA]::[64 bits]

So, the Springfield POP would look like:

  2001:DB8:1A00:: /48  -- Reserved

  2001:DB8:1A01:: /48  -- Springfield Customer 1

  2001:DB8:1A02:: /48  -- Springfield Customer 1

    [...]

  2001:DB8:1AFF:: /48  -- Springfield Customer 255

Similarly, the Winterfield POP would look like:

  2001:DB8:1B00:: /48  -- Reserved

  2001:DB8:1B01:: /48  -- Winterfield Customer 1

  2001:DB8:1B02:: /48  -- Winterfield Customer 1

    [...]

  2001:DB8:1BFF:: /48  -- Winterfield Customer 255

So, every POP can have 255 customers with a /48 each.

Infrastructure POP

We'll reserve the initial POP customer (00) for infrastructure.  It'll let us have easily remembered and typed addresses for things we might need to hardcode, etc.  When other POPs get customer codes, we'll use them to differentiate things we might find useful.

  2001:DB8:0000:: /48     -- Loopback addresses

  2001:DB8:0001:: /48     -- Pt-to-Pt connectors

       [...]                    [...]

  2001:DB8:000F:: /48     -- [infrastructure addressing]

  2001:DB8:0010:: /48     -- server block 1

       [...]                    [...]

  2001:DB8:001F:: /48     -- server block 16

       [...]                    [...]

  2001:DB8:00F0:: /48     -- NOC testing?

       [...]                    [...]

  2001:DB8:00FF:: /48     -- NOC testing?

Loopback Addresses

Loopbacks will have a special construction, so we can tell which POPs things live in.  The full format will be:

   2001:0DB8:0000:0000:0000:0000:0000:XXYY /128

So, upper bits correspond to:

   POP:   00     -- Infrastructure!

  Cust:   00     -- none / us!

   LAN:   0000   -- first usable LAN

This can be typed as:

   2001:DB8::XXYY/128

        XX = POP code

        YY = device number

This gives us 256 loopback addresses per POP, which should be sufficient for growth.

Infrastructure PT-to-PT links

Pt-to-Pt links will also encode POPs in them, and have the full format of:

   2001:0DB8:0001:XXYY:0000:0000:0000:000Z /127

Shorter typing version:

   2001:DB8:1:XXYY::Z   /127

     XX = POP1

     YY = POP2

      Z = Interface number

We can make these /126 if we want to be able to have other equipment addressable on these links.

Customer PT-to-PT links

We can carve customer Pt-to-Pt links out of the reserved block for each

POPs' /40.  It would look like this for the Springfield POP:

  2001:DB8:1A00:: /48  -- Reserved

  2001:DB8:1A00:00YY::Z /127

      YY = POP Customer

       Z = Interface number

Life is change...

I used to think I was a Network Engineer.  Recently, I realized that the best work I was doing was only tangentially related to networks.  It was a revelation.  As friends pointed out, I do "interesting things."

Originally, this blog talked about ways to incrementally improve our (USA) legal/legislative system.  I referred to it as:

Some short rants that just might help. But probably not.

Maybe I'll rant, or perhaps not.  I will, however, start documenting more of what I do, because people have asked me to do so.

The first new post comes from a request to publish an IPv6 numbering plan that I put together.  Others will follow.

Simplified legal system via drown 'em in votes

In 1911, the number of seats in the House of Representatives was fixed at 435. With a population near 97.2 million at the time, the number of people a single House member represented was approximately 223,400. The population has now grown such that a single House member represents more than 716,000 people.

Article I, section 2 of the US Constitution indicates that this number should not drop below 30,000.

Let's go back to the 1911 ratios.

This suggest would amplify the effect of the thoughts at:

Simplified legal system via staying at home.

Simplified legal system via enlightened self-interest

(Unabashedly lifting thoughts from Robert A. Heinlein...)

Let's have citizens granted the right to vote only after they've served
their country in some fashion for a minimum of two years.

If you don't want to serve, you still get all the other benefits of
citizenship -- you just don't get a chance to vote yourself bread and circuses.

Please read Starship Troopers by Heinlein if you need more information
about the pros and cons of this suggestion.

Simplified legal system via staying at home

Let's have Congress meet electronically the majority of the time. Modern
communications can allow most legislation to occur "back home."

• Constituents would have better access to their law makers, and much better visibility of them as well.
  
  
• Outside influences (lobbyists, etc.) would not be able to concentrate on a single location to have sway over several law makers.
  

Simplified legal system via mandatory sunset clauses

I purpose a mandatory sunset clause for all laws passed by legislators.  The length of the sunset period would be related to the margin by which any given law is passed.

Any law supported by more legislators would remain "on the books" longer than those that are passed by small majorities.  Over time, the laws that most influence our society would become laws supported by a large majority of society.