Shmoocon Labs 2012 – Part 4 – Liam «

Here is Part 4 on our series on the experience of helping run the labs network at Shmoocon 2012. This post was written by Liam Randall, aka @Hectaman Let us know what you think in the comments!

Shmoocon Labs Reflection

This was my first year at Shmoocon and I went ahead and decided to make the most of my trip out to Washington and join the labs crew for the network build out and management.  I was not disappointed- here is my story…

Selected as a Shmoocon labs participant your challenge, should you choose to accept to it, is to assemble, manage, and maintain a secure QOS enabled network to operate all of the services required by the conference.  Your “client” population includes the physically distributed conference presentation and facilities themselves, a worldwide live streaming audience, a variety of vendors, sales droids and the “PCI Secured Network” required by their POS terminals, the capture the flag competition, secure/public/guest wireless network partitions, the core management services and let me not forget 1600 of the worlds’ most creatively mischievous hackers, crackers, coders, pen testers and all around rouges.  Did I leave anyone out?

Your resources include an assorted collection of servers, an upstream pipe, the wifi gear in your garage, a couple spools of Cat5e cable, some pretty cool gear on loan from vendors anxious to prove their equipment in a competitively and cruelly Darwinian network ecosystem, all of the open source software you can download and the collective wits and experience under the core leadership of crack team of volunteer staff professionals.  This being my first Shmoocon labs, I was a little unclear as to even what to expect.  While there was some loose organization on the mailing list in advance I found that, arriving late on Wednesday night, much of the core labs team sharing beers and broadly reviewing the pros and cons from previous years labs setup.  Over a few pints they shared stories of firewalls melting their own ASICs under the crushing load of hoards of mischievous attendees.

In advance, the labs participants were organized loosely into groups responsible for provisioning and configuring a core network service.  The teams were wireless, physical networking, firewall, IDS/Monitoring, visualizations, and core (dhcp, dns, etc).  If you came by the lab I’m sure you saw the different network teams loosely clustered around the room- it may have looked a little chaotic, but in less than 36 hours we turned up a first rate network.

Each team was under the leadership of at least one seasoned networking professional who had prepared and was ultimately responsible for delivering for their team.  In short- each and every one of the team leaders was elite.  There were too many to call out, however, on the wireless team, for example, you had not only Rick Farina (zero_chaos) of Airtight Networks and AirCrack-NG fame but Mike Kershaw (dragorn) the author of Kismet.  On networking, Enterasys not only sent a pallet of first rate networking gear but a brilliant solutions engineer Matthew Humm (find the official Enterasys recap here) to professional and securely stich it all together.  Palo Alto Networks sent over a box load of their latest greatest gear and a couple of technicians led by Maher Ghazzi to round out the team.

If you’ve worked on an intense project with firm deadline you’re probably already familiar with the high paced performance demanded out of each team member- and Shmoocon labs was no different.  Along with an incredible opportunity work side by side and learn from internationally recognized experts in their chosen field  we also had plenty of good natured joking- Shmoocon labs was an incredible experience!

There were too many memorable and hilarious stories to share, like the time we pointed the two separate Wireless Intrusion Dection Systems (WIDS) at each other and told each of them that the other was the enemy, however I think sharing some stories from the networking team would better illustrate my point.

I had personally volunteered for and worked with JP Bourget of RIT and the wonderful blog.syncurity.net blog operating our IDS and monitoring, however every team kept on gravitating back to the networking team- initially we all needed Layer 2 to start testing out and tying together all of our network gear.  At first some of the team members were good naturedly ribbing Matthew Humm that he knew so much about networking “he had invented the link light.”  Even on the first night over a Guinness, I sat around a table and enthusiastically discussed what we found to be the least understood components of our  clients networks- I, along with a few others initially argued that QOS would probably be the least understood when Matt chimed in firmly that no, it’s Spanning Tree Protocol (STP).  It only took his brief outline of his thesis of why STP was the least understood networking protocol to start to agree with him.

I did not poll everyone, however to lend a little perspective to this discussion, the group of us sitting around the table had I would guestimate an average of 20 years of serious experience under their belts.  Most of us had lived and worked through the dot-com bubble and had taken turns at many of the worlds’ most recognizable brands and companies.  To date ourselves further, most of us had actually routed IPX/SPX at one point in our careers so this wasn’t a bunch of paper certs sitting around talking shop.  And when Matt started talking we all started listening.

Back to the lab- the core was lit.  VMs were provisioned.  Recipes were incanted.  VLANs were tagged and ACLs were tested.  Even before our special friends started queuing up down the hall we penetrated ourselves (deliberately NOT linked .  We setup services and destroyed them in a hail of malformed packets.   QOS was configured, beat to crap and configured again.  The team leaders zeroed right in on the core services and tested them in explicit detail.  Late into the evening on day 1 we were close- there were still some loose ends to tie up, but the network was essentially configured.

As Shmoocon labs go though, we continued to banter about everything and eventually convinced Matt of Enterasys to give us that mini lecture.  Don’t get me wrong- there were hundreds of lab stories that are relevant and would be fun to tell; I choose this one because it illustrates a further point.  There are some synergies that are only possible under the right set of circumstances- where else would you even find a group of people interested in a masters level dissertation on the intricacies of STP including the key differences between vendors’ implementations, the standards, and all of the major topics- operation, implementation, security, and so forth.  With nothing but a whiteboard and slice of pizza Matt gave two essentially impromptu and lightly scheduled lectures on STP; they were both amazing.  By the end of the first lecture random conference participants had started to join us queuing up and filling the room to what imagine to be it’s legal capacity, perhaps even beyond it.

Take this experience and repeat it.  Repeat it with each of the vendors, each of the team leaders, and add it to the firehose of knowledge normally consumed in con-time at con-speed.  It was enriching and humbling at the same time; participating in labs was an honor.  I could go on- I am new to the “con” scene but somewhat of an old hat in the field.  I don’t think that I possess to the capacity to communicate what an incredible experience Shmoocon labs really was- I leave labs a bit smarter, with loads of new friends, and anxious to F5 way back there in 2013.

Liam Randall
Follow me on Twitter, @Hectaman

Thanks Liam! I can’t wait for next year!!!