Low Bandwidth 101

Back in 2024 I started in ernest building a Proxmox cluster within an HP5400zl series switch. and had continued continued tweaking it in 2025. However, I kept having system lock-ups on boot, particularly during the phase where the LACP bonds were brought online. And while fun and functional, the venerable Core2 Duo CPU just doesn't support enough RAM for some of the applications I would like to run, nor does it have the horsepower to handle traffic much beyond 1Gbps.

So, despite the rising materials costs, I am in the process of fully demoting all the Core2 Duo nodes out of the cluster and maybe even retire the V2 i7 module out as well. The new setup still incorporates a 5406zl switch as managed POE++ hardware and power efficient 25G is still hard to come by.

The new core 5406zl setup is currently:

Slot	Module	Slot	Module
A	HP J9857A Adv Svs v2 zl Module	B	HP J9536A 20p GT PoE+/2p SFP+
C		D	HP J9536A 20p GT PoE+/2p SFP+
E	HP J9536A 20p GT PoE+/2p SFP+	F	HP J9536A 20p GT PoE+/2p SFP+

with three Dell VEP-4600/SD-WAN Edge appliances uplinked via 2x 10GigE in LACP

• SD-WAN Edge 3800, upgraded w/extra 4TB SSD and the 2x10G/1x1G replaced with 2x25G rNDC


• SD-WAN Edge 3400, upgraded w/extra 4TB SSD and the 2x10G/1x1G replaced with 2x25G rNDC


• SD-WAN Edge 3400, upgraded w/extra 4TB SSD and the 2x10G/1x1G replaced with 2x25G rNDC

( more details and some troubleshooting for the remaining ZL compute module )
While I'm hoping the changes will make the v2 ZL Compute module stable enough to leave in production, I'm not holding my breath about it. In the meantime, the extra horsepower per watt of the VEP-4600 systems is already letting me set up some other ideas I've been sitting on for quite a while.

I just made the mistake of flipping into the past here ... and younger me had some things to say that I don't agree with now. Or now appear to be just cringe.

Just a reminder, opinions and ideas can change with time and input. What someone posts on the Internet today isn't necessarily what they'll believe in 3 months, 3 years, or 30 years.

Now to find time to rant about all the censorship, age gating and genAI BS that's taking over the Internet. *sigh*

I've wanted to make more posts here, but so many things since 11/6/2024 have been ... the bad kind of chaos.




In part due to that chaos, I've decided to yeet the hosting for all the old images on this blog. While it makes the archivist and historian in me sad , the company I've been hosting with seems to have jumped on the "AI" bandwagon for a quick buck, so I no longer wish to support them.

I have a much longer post that's been percolating on the entire modern "AI" landscape, but I haven't had the energy and time to start that rant with everything else going on in the world.

To everyone stateside that might see this, please keep fighting for making the world a better place. Resist being battered down by the insanity that the two-party system has wrought upon us.

I've been reading and posting about webcomics for a long time, and decided June was a good month to revisit the list, do some pruning, maybe even take some notes.

1. Dominic Deegan is what started me down the webcomic rabbit hole around 2004. Reading the archive on 56k was paingful but worth it.


2. The Devils Panties I think I started reading around 2007.


3. Sinfest is still on the list mainly because it provides a window to those outside my normal sphere.


4. Least I Could Do , not sure what originally drew me in, but I've probably been reading it for over a decade.


5. Unshelved is just re-runs now, but it reminds me of all the books I want to check out some day.


6. Dumbing of Age I started reading after


7. Shortpacked ended it's official run as a comic.


8. Misfile is another comic I've been following for a long time. It's an interesting perspective on identity involving Angels, Demons and racing.


9. Sluggy Freelance is a Fantasy/Sci-Fi romp on it's 26th epic year.


10. Doc Rat a comedic and contemplative medical almost daily from the land down under.


11. Wapsi Square Fun paranimal characters going through learning how to blend in with the rest of the world while not ending the universe.


12. Girls with Slingshots Currently getting the full-color treatment in re-runs, but still a fun story about a talking cactus.


13. Questionable Content Robots, Friendship and romance since 2003. I think I started reading around 2005 or 2006.


14. Sister Claire The old tag line of "A pregnant nun, holy crap" sums up the first major arcs nicely. A Fantasy romp through a complex and fun world.


15. Spinnterette Ohio's resident multi-armed superheroine and friends!


16. Funny Farm , read the entire original run, kind of forgot about the reboot. Catching up again now.


17. Accursed Dragonong running fantasy story about a guy turning into a dragon & friends.


18. CommitStrip Programming & Geek humor, updates have been very infrequent the past couple of years.


19. Devilbear (Shorts) Monday-Thursday single frame gags set in a cuddlier version of Hell.


20. NSFW Grinder$ a weekday(?) webcomic about a Bakini Barista with a sex and ethics problem.


21. Spacetrawler an epic sci-fi romp of incompetence for good on it's 4th major series of Events.


22. 21st Century Fox, Anthromophic Sci-Fi comic following the tale of the fox Jack Black and his friends & family. Packed with references both blatant and obscure.


23. Mystery Babylon A weird look at the Book of Revelations and other religions with an action/comedy flair.


24. NSFW What happens when you pair the worlds Sexiest and Smartest Secret Agent with the world's most incompetent boss? Spying with Lana


25. The Storm Stained & Dances w/Death Fae follow faith to distaster, then revolution!


26. The Draconia Chronicles Dragons and Tigers have been at war for eons, out to decimate the other side! Will one become the victor, or will the Foxes keep the battle going forever?


27. Alien Dice A long running series about Chelsea falling for an alien named Shira and getting wrapped up in a high stakes game involving powerful corporations.


28. El Goonish Shive The author sums it well: "A strange comic about a group of teenagers and the bizarre, often supernatural, situations that they face. Includes a continuing storyline with non-linear joke comics on the side. WARNING: Often ignores the laws of Physics."


29. General Protection Fault An epic tale of a small software company entangled in the affairs of the multiverse!


30. Penny Arcade

Thirty Comics for the 30 days of June!

A "fun" part of the version 1 ZL compute modules ( j9154a ) is that upon installation the 10GigE ports will always have 100kbps inbound rate limiting set. This creates a lot of headaches when you're trying to use them unofficially for high-throughput firewalls.

HP appears to have some daemons that communication through a GPIO or serial connection to the chassis with their official builds. From what I can tell, they have the super-creative names of
ESPb
ESPb-becket
espd
monitord (?)
However, I have yet to figure out how they interact with the rest of the system.
So I looked for alternatives. First I considered using expect scripts to log into the switch and run the appropriate CLI commands each time, but that seemed possibly error prone.

However, Despite HP making it nearly impossible to download the MIB files from their website, I found some online resources that allowed browsing the HP-ICF-RATE-LIMIT-MIB and HP-SWITCH-BASIC-CONFIG-MIB.
By using SNMPwalk, I determined that my module ports in slot C was .49 & .50 on the hpIngressRateLimitPortControlMode leaf needed to be set to an integer value of 1.

Some more searching to save the config via hpSwitchSaveConfig = 2 and we have a fun little one liner that makes the appropriate changes via SNMP!
snmpset -v1 -c public 192.168.XXX.YYY .1.3.6.1.4.1.11.2.14.10.2.14.1.5.1.2.1.2.49 i 1 .1.3.6.1.4.1.11.2.14.10.2.14.1.5.1.2.1.2.50 i 1 1.3.6.1.4.1.11.2.14.11.5.1.7.1.29.1.1.0 i 2

An additional note is that if the switch has decided the module has failed to boot instead of "booting timed out" it will periodically re-add the rate limiting configuration on it's own. If it boots normally or times out, it will leave the non-rate-limited configuration in place.

While I would like to figure out the in-band method of changing the settings/telling the chassis life is good, the SNMP method will at least allow me to write a startup script that can fix the setting on each module boot.

Update : If the module ports are configured as a trunk port, the rate limiting is automatically suspended, even if the port has rate limiting enabled. HP Manuals, All traffic rate-limiting I think I will leave the scripts in place in case I decide to remove the LAG at some port in the future.

When I started these sporadic writings years ago, never did I imagine I would be a parental unit. I was far more worried about what was in the next issue of Computer Shopper, whether or not I'd actually be graduating high school, and what the mechanical team had screwed up this time on the robot.

Not that long ago, I was surprisingly content with a tiny pool of friends, a large apartment to myself and multiple active Eve Online subscriptions. Then I started dating and met some amazing people, most of whom I'm friends with to this day. Then, COVID-19 happened, one of those amazing people moved in with me and I bought the house I've wanted a few months later. Fast forward two more years, and we ended up getting married ( hoorary for health insurance and taxes ). Fast forward another 11 months and we became parents.

Now I sit, realizing I have a two month old hooman spawn and hopefully 18+ years of obligations to it. It's been a weird change, but in some ways a pleasant one. Renovations that were getting pushed back by "we want to do X this weekened" or " I worked 80 hrs and I'm le tired" finally got mostly done. We have something to distract the family from awkward political discussions. We have a cute little cuddle-bug to ruin in our own unique way. XD

It's a big change, and not for everyone. It's scary and exciting at the same time.

Anyway ... back to preparing for geekly things like yearly backups and LANCLAN 2023!

I had parts to upgrade my old FreeNAS system from a Core-2 Based Xeon on the Intel S5000PSL board to what I was planning on using as a Proxmox + VFIO box using a Tyan S7012 board based system. However, that plan stalled hard when I found out that the Tyan S7012 board does not have proper IOMMU management in the Bios.

After letting it sit through a busy 2021, and a Reddit post about the LinkReal LRST9608, I decided to just install TrueNAS on the S7012 and see how awkward BHYVE really is. After waiting less time than expected to get the LinkReal card off NewEgg, I ran into another surprise with the S7012 board. As the Linkreal card would just barely squeak by the SGI/Rackable IO panel covering up the external access to the lowest slot on the board, I tried to install it there as the boot device with 2x 250GB SATA M.2 and 2x 2TB 3.5" SATA HDDs attached. But the Boot Rom Wouldn't load. After filing a support ticket with LinkReal and doing another weekend worth of tinkering, it turns out you have to Enable boot support for the optional on-board LSI SAS controller of the S7012 for it to allow boot roms in PCIE Slot 1 to actually load. Once I figured that out, I used Ventoy to set up a boot drive for the TrueNAS installer and installed the OS to the SSD pair.

Into the old S7012 SGI/Rackable based beast, I also installed a SolarFlare 10G SFP+ card, an LSI external HBA and a 4GBps Fibre Channel controller. Off the LSI HBA is an SGI/Rackable DAS that the previous owner had upgraded to a 6Gbps SAS expander. Migrating the disks from the old Core-2 Xeon based Rackable Running FreeNAS led to a minor scare, when the first time I tried importing the 8-drive ZFS array it would only auto-detect half the disks as being part of it. However, after re-importing and re-exporting them from the old machine everything worked as expected the second time. The smaller array of 4x2TB Hitachi's I've had forever now exported and imported without incident.

The FreeNAS box was originally set up to be an AD Domain Controller as well, and a combination of the feature being demoted, TLD rules getting changed and liking the additional granularity that AD based permissions gave me, I decided to build a new domain to go with the new host.

First, on my HP 5406zl Advanced Services Module Proxmox host I set up a Debian Bullseye VM to be my first AD controller following the gist of the server-world.info Debian Buster tutorial.

Next, I set up a Windows 7 VM on the TrueNAS host in BHYVE to act as an RSAT client. That was a bit of a challenge in a couple of regards. Firstly, the Windows 7 installer would only complete if I installed using UEFI, virtio storage and virtio networking. In order to do this, I had to add both the Windows 7 installer and the virtio drivers ISO to the VM during setup. The second issue is the mouse never wanted to work in the noVNC session, but some keyboard gymnastics later I was able to get RDP enabled and start working on the rest of the setup. Third issue was that Microsoft had pulled the Windows 7 RSAT tools off their website. Fortunately, there were people that had mirrored them and I was able to get them installed as well. Finally, I joined the VM to my shiny new domain and made sure everything worked (User and OU creation, etc.)

However, Active Directory is something you don't want running on only one host on your network if at all possible. So, I set about adding a second SAMBA-4 domain controller. This is a bit tricky as the SAMBA team hasn't had the time/resources to get DFS style replication working for everything yet. So I set about getting another Debian Bullseye VM working, this time on BHYVE. However, the installer launched into unusable display corruption on the noVNC session. The work around I ended up using was to install Debian using a serial console only, then complete the basic provisioning via ssh.

Then, the first time I tried to add the second domain controller, the basic synchronization wouldn't run correctly between the two DCs. Well... bollocks. Fortunately, I had made a snapshot of all three VMs ( first DC, Windows 7 RSAT, second DC) before I had tried promoting/joining them to the domain. I rolled everything back, tried again and this time everything worked. After that, I went through the process of getting Unison based Sysvol Replication working and it was off to the races.

Finally, I joined the TrueNAS host to the new, replicating domain. I created a service user in RSAT, went through the TrueNAS wizard and a few moments later life was good. It took some time to purge and re-set up the ACLs on the TrueNAS box from the old domain, but it was certainly less painful than some of the Windows-to-Windows migrations I've done in the past.

Now I just have to finished sorting through files, tweaking the shares permissions and set things up to communicate with it again. But at least the big hardware hurdle of the project has been crossed off the list.

It's the time of year that in the US people have to figure out whether or not they owe olde Uncle Sam. Unfortunately, as a linux user, most personal tax software won't run or isn't supported. Fortunately, several years ago I made a Mac OSX VM just for tax time. Unfortunately, this year Mac OS High Sierra is no longer supported by TurboTax. So, I took this snowy new England day to upgrade from High Sierra to Mojave.

1. Take a snapshot/backup of the VM


2. Download a full copy of Mac OS Mojave in your VM. Dosdude1's patcher is great for this.


3. Make an iso of the installer, then copy it to your host machine.


4. Start the install process, expect it to not continue the installer after reboot.


5. Reboot and swear repeatedly as it fails to recognize the Esc key for the 20th time to get into the EFI Menu.


6. Go into the EFI shell, navigate to \macOS Install Data\Locked Files\Boot Files\


7. Run the boot.efi


8. Twiddle your thumbs or listen to a podcast while waiting for Mojave to Actually install..


9. Curse some more when upgrade fails...


10. Figure out the error about being unable to delete file vm due to permissions is a red herring for the installer being unable to update UEFI settings correctly due to the root partition now being APFS


11. Blow off some steam in giant stompy robots


12. Find a usable shim online to boot the upgrade


13. Complete the upgrade, remove extra disks and realize you have to boot from the installation CD and edit the EFI partition contents for a longer term fix


14. Remember belatedly that even with the shims, the tutorial doesn't mention that the path is actually FS0:\GUIDSystem\Library\CoreServices\ and fix it from the EFI cli as vim is not intuitive for casual use.


15. Reboot several more times to verify it's actually working reliably

Update your rant/post on the off chance it gets indexed and proves useful to others.

Internet Archive is now backing up flash animations and using the Ruffle emulator to preserve them in all of their glory.




Something good came of 2020 at least :D

I recently replaced my venerable HP 2848 gigabit ethernet switch that operated as my primary LAN switch with an HP 5406zl chassis switch. I picked the 5400 series due to the availability of 10GigE and POE, plus the availability of services modules.

I ended up picking up a pair of HP J9857A Advanced services V2 modules. These modules feature a quad-core i7-3612QE, 16GB of DDR3 ECC, 2x SD card slots (USB 2.0), a 1TB SATA HDD, 2x 10GigE (internal), 2x 1GigE (1 internal, 1 external). My hope is to get Proxmox running on both of them in a cluster.

So far, I've found out a few interesting things:
- The HP Service OS on the included 4GB SD card is actually a Debian Linux live environment. The card is two partitions with a Primary and Secondary boot.
- You can boot a standard modern 64-bit Debian from an SD card by running the " services # boot service " command
- /dev/ttyS0 at 9600 connects to the serial port for the " services # serial " command

It looks like it might be a rather successful experiment eventually.