OSTEP – The city of ARMs – Building a simple network

After having received the opportunity of joining the Seneca Center for the Development of Open Technology (CDOT) I have begun a journey into the world of ARM architecture which is the focus of the OSTEP team that I am apart of. Following the training for new additions to the team like myself, we were to assign ourselves an initial task to handle. For me that task would be to work with an ARM machine which I’ll lovingly dub Sunny.

The main points of my task is to do the initial set up with Sunny and then to test it with a particular brew of Linux for aarch64 systems to see how stable Sunny is in conjunction with this operating system.

The Set Up

Initial Stage

The first step towards setting up was to understand what manner of connections I could use to work with Sunny from my workstation. Sunny doesn’t have any sort of video ports that one can hook up to a display. The main ports of interest are serial, Ethernet and a SD-card slot. The serial port is how we’ll be interacting with it. With the use of a serial to USB adapter one can use a terminal emulation tool such as Screen, Minicom or putty to connect via the serial port. Directions to using these tools to connect to serial can be found here. Here after I began to familiarize myself with Sunny and proceeded to think about what method of booting I can implement to most effectively place another operating system onto it.

As stated prior aside from the serial, there are Ethernet ports and a SD-card slot. The SD slot is seemingly meant for the use of firmware updates (and perhaps as a last resort to move files to and from). That leaves the Ethernet remaining which then means a network installation (PXE Boot) would be the best choice. Thankfully here at CDOT we have a lovely cabinet of goodies known as the EHL (Enterprise Hyperscale Lab). So instead go through the process of setting up a new PXE Boot server and all the things that it entails, I can leverage our existing infrastructure to make this task much easier.

Secondary Stage

With an inkling of what I need to do, now is the time to figure out how to do it. Certainly I want to be able to use our existing network but how would I do it. Sticking Sunny into the EHL is a little difficult at this time as it’s undergoing some clean up as well as being generally quite full of things already. As luck would have it, we in the OSTEP team are intending to build a second cabinet which might be called the EHL2 but at the time of this writing, the physical cabinet to hold it all together has yet to arrive. However all the other parts that would be come the EHL2 are already present for use. Thus my task then becomes to set up a temporary “cabinet 2” to which will connect to Sunny as well as the original EHL to make use of what’s already here. I think a visual will better show off what is being put together: A pile of gear

This set up is much like a very simple and stripped down version of the actual EHL. Following physically connecting up all the hardware, it was time to configure some of the pieces. The steps to do so were as follows:

  1. Edit the DHCPD config to add new records for the Power Distribution Unit (PDU), Terminal Server (TS) and Sunny. Sunny’s record will also contain the information regarding what it will attempt to PXE boot from.
  2. Power up the TS and PDU and confirm their connection.
  3. Configure TS and PDU as needed. The TS is what allows us to remotely connect to Sunny’s serial port. The PDU will allow us remote power management of our “cabinet 2” network. For example we can remotely power cycle Sunny through the terminal or a web interface for the PDU. Power cycling is the equivalent of unplugging the power cord and plugging it back in again.

Now that we’ve put everything together and configured it, with some luck our system will be up and fully running.

Things aren’t quite working yet stage

Here comes the trouble: For some reason, everything was working except for Sunny’s ethernet ports. Attempting to run a ifconfig command showed that the ethernet devices were detected and enabled but weren’t actually up and running. Previously before I was assigned to this machine there had been a myriad of magical and mysterious issues pertaining to the ethernet but at some point had resolved it self. And once again it appears when it’s in my ownership. Upon the suggestion of my peers, I went to test the power supply in the case and sure enough it was reading as faulty with a power supply tester. After switching it up, everything now works. Hurrah.


Some of the things I’ve learned this time:

  • The things that can make up a server cabinet and how to put one together physically and as well as the configuration that is needed to make it work.
  • Thing to look out for when trouble shooting hardware such as faulty power supplies.
  • Sometimes things will just be a mystery that requires testing things out one by one.

We’re finally ready to PXE Boot but we’ll continue that in another post.

Leave a comment