Supercomputer.io…it’s alive!

During the Parallella Technical Conference on May 30th we ran the first official live test of Supercomputer.io. The whole experiment has come together very quickly over the last few weeks and it’s amazing that we had close to 2,500 CPU cores up and running within one week of launch. The successful first run is a huge testament to the hard work, skills, platform of Resio.io who built the infrastructure and Ola who pulled together the FFT demo application in a short order. As you can see in the video, Ola ran a live comparison between a test image and a library of name associated images to find the name of the person in the test image. The great news is it worked! Clearly, we need to fix some performance and stability issues with the system but this is a big first step to show that these kinds of distributed low power computers can do useful work.

Why this application?

The importance of FFTs in image processing and machine learning is well established. See this paper from Yann LeCun for example. (For all of you who are image processing experts, yeah we know that this is not a practical implementation and this is not the way you would do it in a production system.) Still, as an indication of what is possible, we think it’s a pretty good demo.

 

image_corr.jpeg

Sources

If you are interested in taking a closer look at the application that ran, you can find all sources on github:

Now What?

Together with Resin.IO we have shown that it’s possible to crowd source a virtual computer in short order (I am definitely not calling it a supercomputer yet…). However, the BIG question still remains:

If supercomputer.io can grow to 10,000 or 1,000,0000  CPU cores, what should it be used for?

We need some scientists stand up and say, “Yes, I need this!” If nobody stands up with an application in hand, then we will need to shut it down eventually…

Sincerely,

Andreas

15 Comments

  • Pum says:

    The sound isn’t great but I get the gist. Kudos for the experiment. My parallella first showed ‘idle’ status and has been showing ‘downloading’ status for the last four days, with 0 images processed. Is it not working? Am I doing it wrong?

    • Is it possible that your SD card is read-only?

      • Tom says:

        I connected my parallela on May 30, and I see the same thing (0 images). My SD card isn’t read-only because I simply burned the img and then booted it up. I was thinking that maybe it was because it was 1 hour into the start of the parallela session in the conference.

        Currently the supercomputer.io/#/dashboard says 97 devices online and 0 images per device. Maybe someone can run a test job and let us know if our devices are ok?

  • Rolf Andersson says:

    Hmmm

    I have 3 boards connected, but sofar no activity. Should I leave/keep them on?
    Btw how do I shut them down properly?

    Cheers
    Rolf

  • B-J Velthuis says:

    A little shocked to hear Andreas mention the lack of a more positive response on Adapteva’s presentation of the ‘A1’ and his following conclusion. This really great supercomputer.io initiative clearly counters any (transient?) pessimism. Indeed, I would really like to see this initiative succeed! Considering possible costs that may impede with the up-start and long term success of this initiative, perhaps a few suggestions from a supporting fan:

    1) Please add SSH/console login support to the sdcard image, so that supporting Parallella owners do not have to ‘donate’ their Parallella(s) full-time.

    2) Serial port console access during boot would be ideal, so we could monitor and learn from this process of setting up and using parallellas for docker-based applications.

    3) Donating parallellas to ‘Science’ is of course very commendable, but may severy limit the (initial) use of supercomputer.io., to the extent that the intitiative may eventually fail (which nobody wants). Far better prospects for continued success of supercomputer.io, I think, could result from Adapteva, perhaps sponsored by the parallella community, to open up access to, say, donated parallella owners, or indeed, at some stage, everyone. Opening up access could be implemented in different ways, perhaps gradually. Immediate gain that I can see is for Adapteva to also create a parallella community with hands-on experience in docker-based epiphany applications. This could spark new ideas, which will feed back into parallella based examples in the forums and on you.tu.be.
    4) Not sure whether using a larger, say 8GB or 16GB SDcard, could allow for a concurrent, occasional, personal usage of one’s parallella, which is otherwise made available 24/7/365 to the supercomputer.io initiative. Following this train of thought, would it not be great for the parallella community to be able to exchange ideas, program codes, docker images, etcetera, say, via a global file system hosted by supercomputer.io.? Imagine a local login into one’s own parallella at any time, to ‘play around’ and do whatever we bought the parallella board for. Learn, exchange and share via the community supported global file system. Perhaps a web based login into (an allocated part of) supercomputer.io for all those parallella enthusiasts that want to test drive their newly developed (docker-based) program. Perhaps resulting in the next ‘killer’ application that will change the world? I mean, why restrict to ‘scientists’ that may first need to go through a selection procedure by Adapteva. As if you, Andreas, do not have plenty of other things to do already?

    Best wishes and a safe sailing of the rough Seas!

    Berend-Jan

  • Rob says:

    The idea of some type of “online supercomputer organization/service” is intriguing. You sell the hardware and you have a downloadable for a small fee image that when booted into the computer will go online and log/communicate the time/data it is performing as part of a shared system. Your organization of “members online” is tracked at a website somehow measuring “available computing power”. Users then come to you and pay a fee for use of the “cloud supercomputer”. A percentage of Royalties are then distributed to hardware owners based on processing logged and communicated.

    If you could process time lapse pictures taken of a crowded market area and then search the pictures to photo match to a mug shot of a fugitive then that would be very useful to the intelligence/law enforcement community. I think this is a bit much to ask of a single computer in terms of processing power. The analysts who do this sort of work do not have access to a supercomputer.

  • Jubert says:

    Hi,
    Im shutting off 2 of my boards for the mean time..it has been running since saturday and just being idle ever since.
    please do send email if there would be activity or testing schedule so i may turn them on again to join the cause.

    cheers!
    Jubert

  • Todd says:

    The supercomputer.io site doesn’t seem to recognize my boards (Kickstarter models). I put six of them on a switch that is connected to my router (I have ten Parallella boards in total I may add if I can get them working)… the router says the six “PARALLELLA-HDMI-RESIN” boards are attached and has assigned them IP addresses. I powered it all up about 10 hours ago but the board counter on supercomputer.io still hasn’t increased and the search function can’t find my boards. Anybody else had this problem? Suggestions, router settings…?

    Cheerio,
    Todd

  • Maximiliam Luppe says:

    Supercomputer.io… is it dead?

    All the boards are offline, including mine.

  • I am interested in joining this project, but I do not see any activity… Is there a status on supercomputer.io? Any news about when or if it is coming online?

  • Bobofd says:

    I have 4 boards ready to run, connected two at a time, and no activity. powered them down.
    Echoing some of the comments above, I am willing to participate, with a little more info. With the ability to ssh into the resin device, that is what I call them with Resin IO OS running.

    Updates would be nice. a mailing list, some sort of feedback, I hear the sound of one hand clapping for now.
    bob

  • Sean says:

    I would join in that case that:
    1) the supercomputer.io was a “background” job that stopped if/when I have something of my own to run on my parallela
    2) The parallela remained available to me
    3) I was allowed to queue jobs for supercomputer.io, as a reward for offering my parallela to be a part of it

    These would remove the barrier, and offer an incentive.

    The chance to queue my own jobs is a strong draw, as I want to run some deep learning experiments, which chew up a lot of CPU time. I don’t need to run them often, but want a fast turn around, and the computation is well suited to distributing.

    I would be fine with some kind of system where time is allotted based on how much my own parallela has been online and available. Especially if there were a board where we could describe jobs we want to run and ask others to volunteer the CPU hours they themselves have accrued. I can see a community based system like this generating a lot of activity and motivation and interest and some cool results.

Leave a Reply