My Name is Brian and I Build Supercomputers in My Spare Time

Brian Guarraci is a software engineer at Twitter and in his spare time he’s building a Parallella cluster with a design that was inspired by two of the most iconic supercomputers ever made.

When we saw pictures of Brian’s cluster we were impressed and when we shared these with the community, it became apparent that we were not the only ones! It didn’t take long before curiosity got the better of me and I decided to get in touch with Brian to find out more…

Hi Brian, can you tell me about the Parallella cluster you are building.

I’m building a low-power general purpose compute cluster. I want it to be able to take advantage of standard distributed system packages so that there’s a familiar developer model. The Parallella boards are great for computation but since they have relatively limited storage and memory, I added two Intel NUCs. Each NUC has 1x Intel i3, 16GB RAM, 120GB SSD, 802.11ac WiFi and are also pretty low-power. The NUCs run Ubuntu server and are storage hosts and the primary interface to the external world. The system has 8x Parallella boards and a shared gigabit Ethernet switch, giving a peak performance of around 208 GFLOPs.

The physical assembly is inspired by the Cray-1 and Connection Machine supercomputers, also the new Mac Pro. It is 16″ high, with a 12” diameter and 3″ high base. Starting with the original motivation of creating a cooling tower for the 8 Parallella boards (Mac Pro style), the design expanded to also include power transformers, the two NUCs, a gigabit Ethernet switch, an Arduino and LED strips. In the spirit of the Connection Machine (CM-5), there will be 8x Adafruit Neopixel LED strips mounted on the outside of the tower and each with 16 LEDs, which will show the status of the Parallella compute nodes.

Power-wise, the system has 3 transformers: the two stock NUC 19V power supplies and an external 180W 12V power supply for everything else. The entire expected power consumption is about 120W on average: 30W * 2 (NUC) + 5W * 8 (Parallella) + 20W (LEDs).

A 12″ x 3″ PVC tube is used to form the base. The tube has a milled edge to support the laser-cut disc that is the tower base. The tower is a 13″ x 6″ PVC tube with a 140MM fan mounted at the top. Aluminum rails are mounted inside both the base and tower using high-temperature epoxy. The base itself includes the two stock AC-19V NUC transformers, the NUCs themselves, and 2 19v-5V DC-DC converters providing power for the Parallella boards and LEDs.

I spent a lot of time figuring out how best to mount the Parellalla boards while satisfying the space and airflow requirements. The solution I ended up with was to use a gigabit Ethernet switch as a kind of “spine”. In doing so, it was easy to minimize wire lengths, provide power and simplify mounting. On each side of the switch are 4x Parallellas mounted to oxygen-free copper bus bars. The bus bars are then mounted to the switch using 3M industrial Velcro, which is easy to work with, very strong and serves to insulate the bus bars from the metal switch case. The bus bars are subsequently connected to the base power supplies via standard Molex connectors.

Although building a system with a round case is much harder than a rectangular system, it is definitely more satisfying. To my surprise, when I first fired up the tower I was amazed how awesome it looked and started to think about using a clear acrylic tower tube instead.

Will the cluster have any custom hardware extensions or modifications?

Not at the moment. I think the possibly to remove the HDMI controller from the FPGA and to instead have custom logic for specialized hardware compute support is interesting, though.

What applications do you have in mind?

My current plan is to build a familiar compute cluster setup with host HDFS, Redis and ZooKeeper on the NUCs, and then farm out tasks to the Parallella boards. I think there’s a lot of opportunity to try different styles of parallel computing in the system, so I will use it as a platform for experimenting. Practically speaking, I will use the system for machine learning and Hadoop style tasks.

Why is parallel computing important?

For the most part, CPUs aren’t getting faster and the best way to get more done is to compute in parallel. One of the things that’s cool about a Parallella cluster is that you can perform multiple heterogenous parallel computing tasks all at the same time. Some nodes can be doing image processing while other nodes are doing machine learning. It’s a very flexible setup.

How can people follow the progress of your project?

I call this machine the Parallac and I’m planning on documenting any specific details at parallac.org.

It’s an incredibly cool project and thank you for your time, Brian!