Shmoocon Labs 2012 Part 5 – Branson

Posted by punkrokk on February 20, 2012 0 comments

Check out the other Shmoocon Labs 2012 writeups here.

Happy Presidents Day everyone!  Too bad I’m working. Fortunately Branson came at my with his ShmooLabs writeup today! This is a cross post from http://sandsite.org/2012/02/shmoocon-labs-looking-back/ where you can read some of Branson’s great writing on stuff – and he looks great in a kilt! (I personally like the SysAdmin interview questions.) You can follow Branson on Twitter. A big congrats out to Branson on his wedding a while back also! Here you go!

Shmoocon Labs Part 5 by Branson Mathenson

So now that I’ve settled back into my life after the whirlwind that was getting married and 3 weeks later being at Shmoo, I wanted to reflect on my experience this year, as well as look back a bit at what has come before. Labs ( and shmoo ) has been a part of my life for 7 years now, and 6 of them teaching. It’s always fun, always a learning experience, and always something I really look forward to.

What is Labs?

Labs is an environment where we build all the infrastructure for Shmoocon in 24 hours. It’s designed to be a teaching environment at several levels: team-leads teach a specific area, attendees build that area, and everyone must work together ( most times with someone they don’t know ) to bring things together. While it can be a stressful environment, we also have a really good time finding solutions to problems. Getting to work with your peers is one of the best parts of Labs.

History

So I started with labs waayyy back in 2006 as a participant. I had come to the Shmoocon before, having found it on a short list of east-coast security cons. I had tried several others locally including SANS and ShadowCon at Quantico, but this was the first ‘hacker’ con I went to. What fun. I had never been in a place where people throw balls at speakers for spouting merde, let alone building 2 cycle shmooball shooting devices ( And starting them up in the conference ball room! ). At the end of that ‘con I asked if I could help out. Heidi recommended that I check out labs, so I put in my paper and was accepted.

We had a ball. We had to build the network from the ground up, literally because we had blank machines. We downloaded the ISOs (cd’s back then ;-) and constructed the machines. I was on the ‘infrastructure’ team.. and we got things running pretty quickly. We even hung a hotel sheet on the wall using gaff-tape and put up a display of our system logs and such. Way fun. The team I worked with was a spread of beginners to senior SA’s, but all had the willingness to learn new stuff and try new things. We had pizza and coffee to keep us going and our network didn’t get hacked, didn’t go down. I walked away with a personal commitment to continue to participate, and perhaps get more involved.

The next year, I offered to ‘teach’ Infrastructure, and was accepted. We got labs going and everything went really well ( Tho I think that’s the year we attempted #openbsd and it exploded in our face). The next day as we were last minute tweaking, I found out I was promoted to shmoocon staff when I was unexpectedly invited to the pre-con staff meeting. What a great feeling! It was really neat to have my hard work recognized and become part of such a neat family. It cemented my commitment to working with the ‘con.

So thru the next several years, I continued to teach Infrastructure. I also started thinking of other things i could do for the ‘con. At the ’0wn the con’ one year, I mentioned how silly it was to have paper reviews when we’re a bunch of computer geeks, and was promptly told “well then fix it!”, and so I created our reviews site. I have also taught a self defense course for geeks ( in CoungNhu karate), and this year I gave my paper ‘TTL of a Penetration‘ which was well accepted.

2012

This year, Labs went even better than expected. It was a bit different in that Brett Thorson staged a bunch of ‘vm’s for us, and we had most all of the configs from last year, so we were able to start from about a 70% complete state. Also my team was made up of senior admins, so we could explore some areas we’d never done before. So we added:

  • A certificate authority
  • A puppet-based VM deployment tool
  • central auth using LDAP
  • central syslogging and nagios ( we didn’t have a monitoring team this year )
  • Trac based Wiki and Ticketing system

And everything mostly went really well. Even with all the pre-event planning on the mailing lists ( a record number of emails this year! ) we still dynamically have to alter plans and come up with solutions. We had a few hiccups .. but that’s normal and part of the plan actually. As we’re a group of people with the same objective, and usually diverse training and capability, someone always brings something new the rest of us can learn from. Our team did an outstanding job this year, and I owe them a debt of gratitude.

We’ve already started planning new concepts and ideas for next year, and as labs seems to grow and improve with every iteration, I expect we’ll actually implement some of them. There seems to be a recurring theme for next year in that we start looking at ‘defense’ as strongly as ‘offense’. Hackers tend to like to find ways to break into things, and admins like for that NOT to happen, so I am hoping to build a new idea into shmoocon that incorporates both ideas. Labs is kinda the epitome of ‘defense’ given our attendee group ;-) and so that crew could be a group to move that idea forward. Time will tell.

If you’re reading this and considering labs, you’ll love it. As a newbie, you’ll get to work with senior people who can teach you both theory and application in building a high-risk network. As a senior person, you’ll get to work with your peers, and play in a really cool environment. You can learn everything from IPv6 to making Cat-5 cables, creating a secure firewall to displaying data in really cool ways. It remains one of the high-points of my year, and something I will always look forward to. I encourage you, if you win that golden ticket, apply for labs and come join us! I promise you won’t be disappointed.

Shmoocon Labs 2012 – Part 4 – Liam

Posted by punkrokk on February 17, 2012 0 comments

Here is Part 4 on our series on the experience of helping run the labs network at Shmoocon 2012. This post was written by Liam Randall, aka @Hectaman Let us know what you think in the comments!

Shmoocon Labs Reflection

This was my first year at Shmoocon and I went ahead and decided to make the most of my trip out to Washington and join the labs crew for the network build out and management.  I was not disappointed- here is my story…

Selected as a Shmoocon labs participant your challenge, should you choose to accept to it, is to assemble, manage, and maintain a secure QOS enabled network to operate all of the services required by the conference.  Your “client” population includes the physically distributed conference presentation and facilities themselves, a worldwide live streaming audience, a variety of vendors, sales droids and the “PCI Secured Network” required by their POS terminals, the capture the flag competition, secure/public/guest wireless network partitions, the core management services and let me not forget 1600 of the worlds’ most creatively mischievous hackers, crackers, coders, pen testers and all around rouges.  Did I leave anyone out?

Your resources include an assorted collection of servers, an upstream pipe, the wifi gear in your garage, a couple spools of Cat5e cable, some pretty cool gear on loan from vendors anxious to prove their equipment in a competitively and cruelly Darwinian network ecosystem, all of the open source software you can download and the collective wits and experience under the core leadership of crack team of volunteer staff professionals.  This being my first Shmoocon labs, I was a little unclear as to even what to expect.  While there was some loose organization on the mailing list in advance I found that, arriving late on Wednesday night, much of the core labs team sharing beers and broadly reviewing the pros and cons from previous years labs setup.  Over a few pints they shared stories of firewalls melting their own ASICs under the crushing load of hoards of mischievous attendees.

In advance, the labs participants were organized loosely into groups responsible for provisioning and configuring a core network service.  The teams were wireless, physical networking, firewall, IDS/Monitoring, visualizations, and core (dhcp, dns, etc).  If you came by the lab I’m sure you saw the different network teams loosely clustered around the room- it may have looked a little chaotic, but in less than 36 hours we turned up a first rate network.

Each team was under the leadership of at least one seasoned networking professional who had prepared and was ultimately responsible for delivering for their team.  In short- each and every one of the team leaders was elite.  There were too many to call out, however, on the wireless team, for example, you had not only Rick Farina (zero_chaos) of Airtight Networks and AirCrack-NG fame but Mike Kershaw (dragorn) the author of Kismet.  On networking, Enterasys not only sent a pallet of first rate networking gear but a brilliant solutions engineer Matthew Humm (find the official Enterasys recap here) to professional and securely stich it all together.  Palo Alto Networks sent over a box load of their latest greatest gear and a couple of technicians led by Maher Ghazzi to round out the team.

If you’ve worked on an intense project with firm deadline you’re probably already familiar with the high paced performance demanded out of each team member- and Shmoocon labs was no different.  Along with an incredible opportunity work side by side and learn from internationally recognized experts in their chosen field  we also had plenty of good natured joking- Shmoocon labs was an incredible experience!

There were too many memorable and hilarious stories to share, like the time we pointed the two separate Wireless Intrusion Dection Systems (WIDS) at each other and told each of them that the other was the enemy, however I think sharing some stories from the networking team would better illustrate my point.

I had personally volunteered for and worked with JP Bourget of RIT and the wonderful syncurity.net blog operating our IDS and monitoring, however every team kept on gravitating back to the networking team- initially we all needed Layer 2 to start testing out and tying together all of our network gear.  At first some of the team members were good naturedly ribbing Matthew Humm that he knew so much about networking “he had invented the link light.”  Even on the first night over a Guinness, I sat around a table and enthusiastically discussed what we found to be the least understood components of our  clients networks- I, along with a few others initially argued that QOS would probably be the least understood when Matt chimed in firmly that no, it’s Spanning Tree Protocol (STP).  It only took his brief outline of his thesis of why STP was the least understood networking protocol to start to agree with him.

I did not poll everyone, however to lend a little perspective to this discussion, the group of us sitting around the table had I would guestimate an average of 20 years of serious experience under their belts.  Most of us had lived and worked through the dot-com bubble and had taken turns at many of the worlds’ most recognizable brands and companies.  To date ourselves further, most of us had actually routed IPX/SPX at one point in our careers so this wasn’t a bunch of paper certs sitting around talking shop.  And when Matt started talking we all started listening.

Back to the lab- the core was lit.  VMs were provisioned.  Recipes were incanted.  VLANs were tagged and ACLs were tested.  Even before our special friends started queuing up down the hall we penetrated ourselves (deliberately NOT linked :) .  We setup services and destroyed them in a hail of malformed packets.   QOS was configured, beat to crap and configured again.  The team leaders zeroed right in on the core services and tested them in explicit detail.  Late into the evening on day 1 we were close- there were still some loose ends to tie up, but the network was essentially configured.

As Shmoocon labs go though, we continued to banter about everything and eventually convinced Matt of Enterasys to give us that mini lecture.  Don’t get me wrong- there were hundreds of lab stories that are relevant and would be fun to tell; I choose this one because it illustrates a further point.  There are some synergies that are only possible under the right set of circumstances- where else would you even find a group of people interested in a masters level dissertation on the intricacies of STP including the key differences between vendors’ implementations, the standards, and all of the major topics- operation, implementation, security, and so forth.  With nothing but a whiteboard and slice of pizza Matt gave two essentially impromptu and lightly scheduled lectures on STP; they were both amazing.  By the end of the first lecture random conference participants had started to join us queuing up and filling the room to what imagine to be it’s legal capacity, perhaps even beyond it.

Take this experience and repeat it.  Repeat it with each of the vendors, each of the team leaders, and add it to the firehose of knowledge normally consumed in con-time at con-speed.  It was enriching and humbling at the same time; participating in labs was an honor.  I could go on- I am new to the “con” scene but somewhat of an old hat in the field.  I don’t think that I possess to the capacity to communicate what an incredible experience Shmoocon labs really was- I leave labs a bit smarter, with loads of new friends, and anxious to F5 way back there in 2013.

Liam Randall
Follow me on Twitter, @Hectaman

 

Thanks Liam! I can’t wait for next year!!!

Shmoocon Labs Part 3 – How I spent my SCL vacation

Posted by punkrokk on February 8, 2012 0 comments

Here is a post from Ray Davidson on his 4th year doing Shmoocon Labs! Thanks Ray! Make sure to check @RayDavidson on twitter! I personally had a great time getting to know Ray a bit better during the con! He good peeps! You should too!

Make sure to check out Shmoocon Labs Part 1 and Part 2!

How I Spent my ShmooCon Labs Vacation by Ray Davidson

This was my 4th year at ShmooCon, and my 4th year in labs. If I had to choose, I think I would give up the formal talks for the ability to work more in labs; it’s way cool. I do labs for (at least) two reasons.

First, I always learn stuff. As it happens, I have a bunch of theoretical knowledge (degrees and certs), some of which is probably true, but that just makes me a paper tiger. There is simply nothing like the real life exercise of putting together a real network, on a short deadline, for a demanding clientele. And there is nothing more demanding than a security conference with a reputation to uphold.

Second, I do it for the experience of community. It is reminiscent of community theater – a bunch of people, many of whom don’t know each other personally, come together with a fairly focussed purpose, a venue and a drop-deadline, and basically “make it up as they go along” – using the collective talent to “put on a show”. There is nothing like the feeling when the overture starts, the curtain goes up, and it’s “magic time” again….<cue the Bugs Bunny/Road Runner theme>

ShmooCon Labs has a crew of a few dozen folks, self-divided into teams. We try to start out with fairly evenly divided teams, but depending on what goes well and what doesn’t, people shift to address needs and their interests. In my case, I’m interested in the entire range of the OSI model, from electrical pulses and frequency waves, to packets and frames, to the crazy people at the upper levels and the apps they use. In the past I’ve worked on upper layers – network monitoring and visualization – so this year I decided to get back to basics and work with the routing and switching team.

My labs experience actually started Wednesday night – 7 of us walked to Buca di Beppo for dinner and because the restaurant wasn’t too crowded, we had dinner in the Pope room. I know there exists a video with comments from the attendees; (ed. note: if you guys really want the video say so in the comments) perhaps it will be linked from this post… We had a great time, “partying with the Pope”.

The next morning we assembled the various teams in the NOC-to-be. The routing/switching team leader was Matt Hum, from Enterasys. He brought a pallet load of Enterasys equipment to be configured and distributed throughout the venue. We had a network architecture from previous cons, including VLAN specs, but we also had more equipment than previous years, so we were able to distribute switches more widely. It was quite an impressive stack of equipment – 2-1U 48 port switches for the core, 2-48 port stackable switches for the distribution layer, 5-24 port switches to split between distribution and edge layer, and a dozen stackable 12 port switches for the edge. The 12 ports were particularly sweet; 1U high and half rack wide, with POE. I know several of us were looking at them with SOHO/home lab use in mind.

Matt did us all a favor by creating the skeleton of a generic .config file for the switches. Enterasys configuration commands are somewhat different from IOS, and Matt was good at pointing out the differences (along with suggestions of how Enterasys had advantages). Our team was under the gun, too, since the other teams were dependent on physical wiring and VLAN/switch configs before they could really get their operations running. Fortunately we were able to get the NOC wired and configured earlier than usual, and things were running smoothly on that front by afternoon, so no one had to stay late at night.

We did learn a lesson – complex passwords are really important, but not *too* complex. There is a happy medium. We had a bit of a scramble on Thursday morning to get things set in the ballrooms for the initial presentations. It was a long run of cat 5 down the hallway to the ballrooms, and we were short on gaff tape and time. And with the new ShmooBall configuration, the 12 port switch in each ballroom was completely full. We did manage to get all the switches wired, taped and configured. In the process, we discovered that those little 12 port switches can not only supply POE to access points, but they can also *run* on POE. Probably not recommended, but it was confusing for a while, and amusing when we figured out what was going on.

To sum up, I had a great time, and I hope to be able to do Labs again next year. I learned more this year than previous years – sobriety probably helps – and I appreciate all the teaching form Matt and Ric F. and everyone else that shared information and community. Can’t wait to see my friends again next year!

Thanks again Ray! Please let me know what you though of Ray’s writeup in the comments! Be nice!

Shmoocon Labs 2012 Part 2

Posted by punkrokk on February 6, 2012 1 comment

So this year I have asked people who have contributed to labs to write up their expereiences. Here is the first one from Alex/@bluejaytkd. I want to thank Alex for taking the time to do this!

If have haven’t listened to Ep1 of the Roc Sec Podcast, a bunch of the core labs guys talked about what goes into preparing and executing Shmoocon Labs.

My (@bluejaytkd) Shmoocon labs experience

Feeling lucky

Sitting at my desk at work on November 1st, I had an off-the-cuff moment and decided to try and get a Shmoocon barcode. I had tried for years and always got overrun in the yearly F5 bash. But I figured, what the heck? To my absolute shock after hitting F5 at high noon, the site asked me how many tickets I wanted! Once my heart rate slowed back down, I ordered my ticket and started making plans to be out of work for a few days in January.

Getting started

Upon getting to Labs on Thursday, I knew we would be setting up the conference networks in a nonstop thrash. I’ll admit that as a first timer I was nervous with how good the crew would be and if I could contribute. I’m a home network enthusiast after all, not a sys admin or network admin professionally. Going out to breakfast with the team I tried to figure out if I was in over my head. There are people in this group who have written tools that many people use daily and know well. However, this did not turn out to be an issue since there was always a cable to run or an interface to learn. Everyone was really friendly and willing to teach new techniques to n00bs. After a short amount of time everyone found a niche to work in and we all settled down to get the network done.

I was assigned to the network monitoring and SIEM teams and worked to help set up our tools on several pieces of hardware provided by other labs attendees or vendors. By the end of the day we had a pretty solid working network and logging ready to go. Now we were transitioning into lab operations mode and looked forward to the next day when the conference would begin.

Getting comfortable

Once I had gotten through day one of the conference, had seen how the technical work of the conference would be handled, and been out to dinner with some of my fellow labs attendees, I was feeling a bit more comfortable and up to speed with how things would go.

On the network monitoring team we used a Linux distribution called Security Onion, which is an installable version of Linux with all of the network security monitoring tools baked in. We were curious to see how this tool would work under the conference traffic load. And this is one of the main ideas of Shmoocon Labs: Try to use whatever tools you want in a truly challenging environment. It was a great way to learn in the deep end for me and see how tools really work under a load. Under Security Onion, we ran Bro IDS, Snort, Snorby, PHPMyAdmin, and Sguil. In the SIEM team, they ran OSSIM and OSSEC as well as  Nitro Security IDS. We watched the dashboards for threats and spent the rest of the time playing with the data and trying different ways of querying our logs and information to see if we could find a new way to look at the traffic we were seeing.

Maximizing the conference

* I did find that as with most conferences, morning is a quiet time and, if you get in at 8 am, you can have some time to work and play with the Labs setup while it is still quiet and not too crazy.

* Labs offers a great learning tool. It is hard to re-create a real world environment to play with at home. At Shmoocon there’s a great playground for learning with no repercussions due to unwanted snooping through data.

* Make time to hang out with your fellow labs attendees. Some of the best conversations, connections and learning that I did at the conference was around a dinner table or over drinks. I plan to keep in contact with everyone I met at labs. This blog post itself came out of the experience!

Looking back

Was my first Shmoocon and Shmoocon labs experience was worth it? Absolutely. You may not get as much time to go to talks (although I saw plenty of them) and you may not get quite as much partying time (although you will get a wrist band to the Saturday party). But how often do you get to help setup and run a network in a hostile environment where you get some new toys to play with and you are encouraged to experiment? I imagine we all would call that a dream scenario. I learned more than I could have imagined over the weekend and am playing with many new tools in my home lab as a result. I would echo what most of my fellow labs alumni said in post-con emails “If I can get a barcode… I’ll definitely be back!”

 

ed. Let us know what you think in the comments, please be nice!

 

Roc2600 Presentation: SSH Overview

Posted by punkrokk on February 3, 2012 0 comments

Here are the slides from my presentation on SSH today at the Rochester 2600 meeting. Best part is I figured out how to use plaintext cipher to study SSH. After I gave the presentation, @antitree gave me some info that I could MITM SSH in order to watch the connection build. See here. There is another SSH MITM tool here.

Download Slides Here: SSH2 Overview

Shmoocon Labs 2012 Part I

Posted by punkrokk on February 3, 2012 0 comments

Here is a great video Brett Thorson over at Compute Cycle did on how he setup IPv6 functionality for the Shmoocon Network. You can follow @computecycle on twitter. This was originally posted on Brett’s Compute Cycle Blog. Check out some of his other videos!

February Rochester 2600

Posted by punkrokk on February 1, 2012 1 comment

This Friday is the monthly Rochester 2600 meeting. The past few months this meeting has been gaining momentum, and there have been some good talks. There are six or more talks slated from my own SSH Protocol Part I talk to some Andriod talks, and a review of 28c3 and Shmoocon. Kevin Mitnick cancelled :( Go here to check out @Antitree’s awesome flyer for Friday’s meeting at 7pm. Hope to see you there!

Roc Sec Podcast Episode 1 – ShmooLabs, Roc OWASP #RocSec

Posted by punkrokk on January 24, 2012 1 comment

Here is it! Episode 0×1! We interview Andrea Cogliati, President of the Rochester OWASP chapter, then talk abit about the impending Shmoo, and then we have a group (Ken Caruso, Brett Thorson, Branson, and JP chat about how Shmoocon Labs came to be and what we look forward to in 2012.

Edit: Please take the Rochester OWASP Annual Survey. Here is the Rochester OWASP Homepage.

PS: yea we’ll have an iTunes link soon!

PSS: I guess I already set iTunes up – search for RocSec on iTunes or click here. 

Podcast: Play in new window | Download

Roc Sec Ep1 Preview – Preparing for Shmoocon and Rochester OWASP

Posted by punkrokk on January 19, 2012 0 comments

So today is one week before Shmoocon Labs starts. I fly to DC next Wednesday!

I thought some of the people in Rochester would be interested in what goes into making Shmoocon Labs a reality — so a few of us got on the horn for a segment about Shmoocon Labs!

We are going to finish recording on Sunday and will be interviewing  Andrea Cogliati, the President of the local OWASP chapter about whats going to be going on in 2012. You can check out their site here: http://bit.ly/AdxtOj

We plan to have the podcast posted before we head out to Shmoocon and if you are lucky by Sunday or Monday! See you all soon!

January 2012 Rochester 2600 Meeting – 5 Speakers — be there!

Posted by punkrokk on January 4, 2012 0 comments

Hey boys and girls, @antitree has released the agenda for the first rochester 2600 meeting this year. We have 5 speakers — and it should be a good one! http://www.rochester2600.com/