March 9, 2016
How Can You Scale It If You Don't Trust It?
When you scale up an infrastructure it is crucial that you can trust
you have the right resources in play, the right code deployed and that
information can only flow in a secure manner. When you scale the
organization, trust is required amongst all of the people responsible
for coding, testing, deploying and managing the applications that
power the business.
With all of the chatter around scaling, you would think someone would
have told you the key ingredient necessary for creating and fostering
the required trust. Unfortunately it is very easy to get to the end
of the diving board, right on the edge of jumping into something like
a hybrid cloud deployment, before you realize you need to figure this
out on your own.
This talk can help. We?ll discuss some concrete ways you can engineer
trust into the system (complete with examples) you are building or
operating so that it works well for cloud-native and legacy
applications. By the end, you?ll have a good idea of the
decision/enforcement points you?ll need to consider to be able to
create a system (and an organization) that can scale.
David is the Technical Evangelist at Apcera. He has spent thirty years
in the systems administration/DevOps/SRE field in large multiplatform
environments including Brandeis University, Cambridge Technology Group,
MIT Media Laboratory and Northeastern University. He is the author of
the O'Reilly Otter book (Automating System Administration with Perl) and
is a frequent invited speaker/organizer for conferences in the field.
David is honored to serve on the USENIX Board of Directors. He prefers
to pronounce Evangelist with a hard 'g'.
April 13, 2016
May 11, 2016
- Wednesday, January 13, 2016
- Transactional System Administration Is Killing Us and Must be Stopped
Thomas A. Limoncelli, SRE at StackOverflow.com
Transactional system administration puts sysadmins in a bad
position. When organizations replace this model with a
service-centric model the company receives better service and
sysadmins gain positive visibility and increase their value.
The transactional model is where customers make requests and system
administrators fulfill them. It is the model of servitude. This has
been the power dynamic for decades. The service-centric model is where
sysadmins maintain the automation that does work rather than doing the
work themselves. Another way to look at it is that the best use of
human labor in an auto factory is not to build cars, but to maintain
the robots that build cars.
Stack Exchange's SRE Team is making strides at minimizing
transactional system administration and, instead, adopting DevOps
practices that create a cooperative relationship with our users. We've
adopted this as our guiding management principle and it has greatly
improved how we get things done.
Tom will discuss the successes and failures of these attempts and
recommend how your organization can adopt this better structure.
Tom is an internationally recognized author, speaker, and system
administrator. His new book, The Practice of Cloud System
Administration, launched last year. His past books include Time
Management for System Administrators (O'Reilly) and The Practice of
System and Network Administration (Pearson). In 2005, he received the
USENIX LISA Outstanding Achievement Award. He works in New York City
at Stack Exchange, home of Careers.Stackoverflow.com, and previously
worked at Google and Bell Labs. His blog is
http://EverythingSysadmin.com and he tweets @YesThatTom.
- Wednesday, December 9, 2015
- LISA Recap
All attendees of LISA 2015 are welcome to share their experiences at
LISA. We will go through the program and people will discuss what they
found interesting, worth taking etc. This is a great way to find out
what you missed in another talk or for non-attendees to determine what
they should watch on the LISA website.
- Wednesday October 14, 2015
- Managing cloud costs and technical requirements when every engineer
is an admin or "Herding Cats in the Cloud" and hope for the future.
How do you keep your sanity when most developers can log in to your
cloud console and create/destroy/manipulate resources? How do you
implement policies and procedures when you have many developers doing
different things with different tooling and the lowest common manager
is the CEO? Traditional methods of creating policies and procedures
managed by a small operations group will not work -- these are
barriers to agility and velocity which industry trends are actively
attacking. Dewey will present a set of practices and tools for
managing DevOps with staff responsibilities ranging from pure
development to pure operations and responsibilities to manage multiple
products experiencing multiple thousands of TPS.
Dewey Sasser is a distributed systems software architect who
accidentally wondered into operations near the beginning of the DevOps
movement. He is currently working as a Cloud Solutions Architect for a
major gaming company.
- Wednesday September 9, 2015
- Why we can't have nice things. A tale of woe and hope for the future.
Pete Cheslock, Threat Stack
Computers are hard, and security is even harder. While you?re building
a bespoke host-based intrusion detection system to monitor for
advanced persistent threats, vulnerabilities are uncovered in
30-year-old core Unix programs. Even worse, the same junior level
operations engineer who can (accidentally) provision thousands of
systems and blow your budget away, is the same person who can make one
small change to a security group which now allows all access to your
The cloud is making it easier than ever to provision systems to meet
your infrastructure needs ? and to do so very quickly. Speed to market
is a major competitive advantage that many companies are leveraging
through the concept of Infrastructure as Code. Provisioning hundreds
or thousands of compute instances in mere minutes is now considered an
everyday activity. Everyone wants to move fast.
The long contested battlefield of remote access to production machines
has only gotten uglier since the rise of The Cloud, which has
obliterated the line between building the system and running the
system. ?Lock out the developers? is not an acceptable policy
anymore. Developers inherently build better systems when they
experience running them.
Continuous Integration. Continuous Deployment. But who (or what) is
continually monitoring the state of your operational security?
We'll discuss the role of security in this new *aaS landscape. We?ll
talk about things to do when you have a dedicated InfoSec team, and
tools you can use when you don?t. We?ll explore what it means to build
in security in the same way you build in quality as part of your
continuous delivery pipelines. And how you can strengthen your
security posture while maintaining your ability to move quickly and
deliver value to your customers.
- Wednesday August 12, 2015
- No Meeting - Summer Vacation
- Wednesday July 8, 2015
- No Meeting
- Wednesday June 10, 2015
- Rocket and the Application Container Specification
The introduction of Rocket and the Application Container Specification
brought the number of the Linux container alternatives to three. In
the six months since that announcement, Rocket has matured quite a
bit. It offers lighter weight containers, simpler image discovery and
tight integration with orchestration systems like Kubernetes. In this
presentation you'll learn the basics of Rocket development, usage and
internals as well as the current state of the Application Container
Specification and alternate implementations of the spec.
Mark Lamourine is currently working at Red Hat as a champion of the
System Administrator in the development and integration of system and
software services. In his 30 years in front of a keyboard his work
experiences range from developing and operating ISP infrastructure
services, to managing software and hardware testing labs in much
smaller companies. Recently he's started writing blogs, book reviews
and articles on new tech in computing. He's an (extremely minor)
contributor to both the Rocket/Appc and Kubernetes projects.
- Wednesday May 13, 2015
- The Future of System Administration (and What You Should Do to Prepare)
Adam Moskowitz, MathWorks
System Administration is changing faster and more radically that at
any time in its history. Like it or not, your job is probably going to
be very different in 10 years (and maybe even in 5 years); come hear
where the field is headed and what you should do if you want to stay
on the leading edge of system administration. (Sneak preview of the
recommendation: "stop being a sysadmin.")
Adam Moskowitz is a Senior Tools Developer at MathWorks in Natick MA;
in the past he has also been a system administrator, and his current
project marries those two disciplines and if it works, will make a
significant change in how system administration is done. Adam has been
in the field since 1978 but claims he keeps working "to keep food in
his puppy's bowl" (and to support his barbecue habit).
- Wednesday, April 8, 2015
- Compliance Double Header
Josh Boon PCIP, Walt Williams
Compliance Soup: PCI DSS Condensed
Speaker: Josh Boon PCIP
An overview of PCI DSS 3.0 requirements and definitions. In this talk
I'll define types of credit card data and the storage requirements of
each, the types of merchants and service providers and the reporting
requirements of each, and summary of the twelve requirements of PCI
compliance. A participant should be able to take away from this talk
the basics of PCI and if it applies take the next steps towards
Compliance Soup: ISO, ISACA, SSAE, NIST and CSA Oh My!
Speaker: Walt Williams
A brief history of the alphabet soup that is security standards,
looking at the goals and scope of each of the commonly sought
certifications, applicability of each, challenges in
achieving/maintaining and the bang for the buck if you will, or are
they worth the time invested. The person shall be able to take away a
comparative understanding of the many standards available in
Josh Boon has worked across many verticals including higher education
and ecommerce primarily as a sysadmin expanding into compliance with a
focus on securing credit card data and personally identifiable
information. He now works for NutraClick in the role as a senior
sysadmin and internal security assessor and consults for other
companies in the PCI and PII space.
Walter Williams served as an infrastructure and security architect at
firms as diverse as GTE Internetworking, State Street Corp, Teradyne,
The Commerce Group and EMC. He has since moved to security management,
where he'd served as at IdentityTruth, Passkey and now manages
security at Lattice Engines. He is an outspoken proponent of design
before build, an advocate of frameworks and standards, and has spoken
at Security B-Sides on risk management as the cornerstone of a
security architecture. His articles on Security and Service Oriented
Architecture have appeared in the Information Security Management
Handbook. He sits on the board of directors for the New England ISSA
chapter and was a member of the program committee for Metricon8. He
has masters degree in Anthropology from Hunter College.
- Wednesday, March 11, 2015
- Hyper-V and Openstack
Peter Pouliot, Microsoft
OpenStack is a rapidly evolving open source cloud computing
infrastructure typically thought of as a Linux centric
technology. Microsoft however has been involved with the OpenStack
from early on ensuring that Hyper-V, Windows and other core Microsoft
technologies interoperate within the OpenStack API framework. This
talk will discuss some key areas of integration between Microsoft
technologies and the OpenStack architecture while additionally
highlighting Microsoft's involvement in the OpenStack project
Development and Continuous Integration cycles.
Peter is the OpenStack subject matter expert within Microsoft as well
as the evangelist for OpenStack awareness and adoption of OpenStack
- Wednesday, February 11, 2015
- Meeting cancelled due to snow
- Wednesday, January 14, 2015
- Radical ideas from The Practice of Cloud System Administration
Tom will highlight some of the most radical ideas from the new book
"The Practice of Cloud System Administration". The book focuses on
"distributed" or "cloud" computing and brings a DevOps/SRE sensibility
to the practice of system administration. Unsatisfied with books that
cover either design or operations in isolation, the authors created
this new reference centered around a comprehensive approach.
Some of the radical advice includes: improving uptime by using cheap
unreliable hardware, why you should crashing servers at random times,
and that you should make peace with outages.
(Note: Tom will not have books for sale at the talk, but he will sign
books you bring, even ones he didn't write. Ebook purchasers will
receive a sticker to put on their Kindle.)
Info about the book can be found at http://the-cloud-book.com
Tom Limoncelli is best know for books like Time Management for System
Administrators (O'Reilly) and The Practice of System and Network
Administration (Addison-Wesley). He works in New York City at Stack
Exchange, home of ServerFault.com and StackOverflow.com. He tweets
@YesThatTom and blogs at http://everythingsysadmin.com
- Wednesday, December 10, 2014
- LISA Recap
Recap of LISA 2014
- Wednesday, November 12, 2014
This date overlaps with LISA. Meeting has been cancelled.
- Wednesday, October 8, 2014
Project Atomic: Server OS and app container delivery with Docker and OSTree
For many years, "traditional packaging" has been the default method
for software delivery and management on Unix systems; exemplified by
dpkg, RPM, and similar systems.
Project Atomic is a pattern that can be applied to a traditional
distribution (for example, Fedora, CentOS, Red Hat Enterprise Linux),
bringing together several upstream components: SELinux, Linux kernel
containers, Docker, (RPM-)OSTree, and orchestration frameworks such as
geard and Kubernetes.
This talk will explore the details of all of these technologies, with
a particular focus on the RPM-OSTree side, which provides atomic
upgrades and rollbacks for bare metal operating systems For example,
we'll look at why /home is a symlink to /var/home on an Atomic system,
and how configuration files in /etc are carried forward on upgrades in
a fully atomic fashion.
Colin Walters is a Senior Software Engineer at Red Hat in the Server
Experience group. He is the upstream author of OSTree, and has
contributed to a variety of Free Software projects, such as Emacs,
systemd, GNOME, OpenEmbedded, and both Debian and Fedora packaging.
- Wednesday, September 10, 2014
Trying to Outpace Log Collection with ELK
This talk will detail the adventures of centralizing log data with the
ELK stack: Elasticsearch, Logstash, and Kibana. We wanted to
centralize logs from many very separate networks because it's a really
good idea to look at logs more than you probably are. The adventures
in this project include building concurrent prototypes with Logstash,
Graylog2, and Splunk. There's a bit of layer 7 routing and buffering
with RabbitMQ. I might get a bit DevOps-y in the description of using
Chef to deploy all the systems in this project. And finally, there are
some lessons learned about running distributed magical databases like
ElasticSearch in Amazon AWS. tldr; Lots of stuff only breaks when it
gets big enough to break.
Neil Schelly has been a sysadmin, developer, or consultant for most of
the last 20 years. Currently, he's a Principle Security Administrator
at Dyn, Inc in Manchester, NH. Lately, there's been more of a
security, auditing, and monitoring focus to the projects he is working
- Wednesday, August 13, 2014
No Meeting - Summer Vacation
- Wednesday, July 9, 2014
Server Names? We don't need names where we're going!
How do you thrive in a environment where you don?t know your servers'
names, or if they'll disappear at any time? How do you build servers
with configuration management in this kind of environment? What do you
give up? This talk will discuss how you can name your hosts to make
life easier, and covers using cloudinit, autoscaling, mcollective,
logstash & sensu.
Matthew Barr has been a sysadmin (professionally) for over 15 years,
managing both cloud based & datacenter infrastructures. He's led the
ops group at Kensho & Snap Interactive, and the US sysops group for
MarkitServ. He recently finished building a production autoscaling
infrastructure in AWS without knowing his hostnames. He's a
co-maintainer on puppet-jenkins, and has contributed code to Puppet,
logstash, and various other projects.
- Wednesday, June 11, 2014
Lay waste to waste
This talk will cover the basic ideas and some background behind Lean
and 6 Sigma (LSS), discuss their origins and then walk through some
examples while introducing the tools and techniques. Plus you get to
test the speaker by bringing your own problems and (hopefully) leave
with some ideas on how to tackle resolving the problem.
John Rouillard has been a system administrator trying to fix practices
for more decades than he cares to remember. His interest in formal
problem solving techniques developed by watching solutions that didn't
really solve problems but merely added more problems while allowing
the original issue to generate even more failures.
- Wednesday, May 14, 2014
IT Project Management
20 years ago SAGE was defining IT job skills sets for HR
departments. Today (no surprise) IT departments are the heart beat of
the enterprise. With burgeoning budgets and even more complex
technologies management is trying to get more of a handle on their IT
processes and align them with standard management
methodologies. Sometimes with good results.
I will touch on TQM, ITIL, PMP & Agile practices and why some
practices work and some do not. Much of it depends on the culture of
the enterprise. Managing projects at large enterprises
(e.g. Microsoft), higher education (e.g. Harvard) and startups is
vastly different and requires a large bag of tricks. A major issue of
project management is who is "The Boss"? Is it the technical lead, a
non-technical manager or the project manager? In IT, it is often the
technical leader that is the real boss, regardless of titles.
Most Project Managers come from a business background. But IT Project
Managers better be able to understand the technical aspects of IT. If
an IT Project Manager asks naive questions like "Do you really need
fail over for DNS?" They will be abused and ignored by the technical
staff. Hence, many IT professionals see Project Managers as someone to
avoid. Management hears "The IT department is not cooperating" from
the Project Management staff.
I will share my horror stories, but actually explain how Good IT Project Management can help your IT department be more successful.
Peg is the Senior Technical Project Manager for Starfish Storage.
- Wednesday, April 9, 2014
The Practical Gamemaster: Design & Execution of IT Emergency Operations Drills
Practical IT drill design brings together emergency response and
operations, business continuity, disaster recovery, and IT
architecture. During this talk, you will learn key concepts in
emergency operations center and incident headquarters design, methods
of introducing such concepts to your organization, and a sequence of
basic-to-advanced drill designs. Keeping IT folks engaged in a drill
simulation can be very challenging. Become a practical gamemaster
worthy of designing and executing drills on likely emergency scenarios
and realistic function failures for your organization. d10s included.
Adele Shakal currently heads up project and knowledge management at
Metacloud, Inc., a cloud solutions company providing on-premises
private cloud based on OpenStack. In her prior work in technical
project management and system administration at USC ITS, she designed
IT emergency operations and the ITS Great Shakeout 2011. She has more
than two decades of experience in information technology, with
Bachelor of Science in geochemistry from California Institute of
Technology. She has presented at CENIC, EDUCAUSE, APRU, USENIX LISA
and CascadiaIT conferences.
- Wednesday, March 12, 2014
"Hi, my name is" Lightning Talks
Adam Moskowitz at al.
- Wednesday, February 12, 2014
Solid State Disks
Solid State Disks are a relatively new technology, but on the surface,
seem to function exactly like traditional hard drives, only
faster. The truth is that the underlying way they store and retrieve
data are completely different - and understanding how and why can make
the difference in making sure that your disks last much longer and
maintain their performance.
This talk is an adaptation of a tutorial presented at LOPSA-East
2013. Because of time constraints, we'll be sticking to the most
relevant and interesting parts, and you will still be able to walk
away with a much better understanding of how to use SSDs in your
environment, and make well-informed purchasing choices.
Matt Simmons is a 13+ year system administrator, blogger, and
community organizer. He's currently serving on the Board of Directors
of LOPSA, and as a program committee co-chair of LISA14. His blog can
be found at http://www.standalone-sysadmin.com.
- Wednesday, January 8, 2014
The role of the central IT department is changing. No longer are users
happy inadequate IT organizations that create solutions that aren't
fully baked out. More executives just want to "throw some cloud on
it." Given the difference of user expectations between internal
services and what the outside appears to offer -- can you blame them?
This talk focuses on how IT lost its edge, and how IT organizations
can work towards being more human-focused and ultimately regarded as
business game-changers and not just a cost center.
Patrick Cable is an infrastructure system administrator at a local
research institute. He's currently working on managing a network used
in hardware design and verification. He also provides guidance to
other internal groups about best practices in system management. Prior
to his current job, he operated an IT consulting business that worked
with a variety of clients - including small and large businesses, and
- Wednesday, December 11, 2013
Project Managers as Value Role in IT
Responding to "urgent/non-important" IT requests takes time away from
developing initiatives, completing complex operations, and evaluating
hardware or software technologies. Given this ever-present aspect of
IT, many departments believe that sysadmin, service desk, network, and
security areas are the obvious priorities when increasing
staff. However, if project management is valued as a staff position or
as a portion of present staffing roles, projects will have the high
visibility and accountability they deserve, plus the resources they
need in order to be Done Right.
The fundamentals of project management (sponsors, stakeholders,
resources, scope, methodology) and how they can be used to drive the
strategic vision of IT departments can be transformative for a
department and its staff.
This talk is not a tutorial about "how to be a project manager",
instead, attendees will hear about how to advocate for and develop
project management within their groups, completing projects despite
the seemingly endless lists of things to do.
Cathleen Bonner has worked in IT at the Broad Institute for over a
decade. She began in desktop support, shifted into research computing
engagement, and now manages the project and portfolio team. Cathleen's
goal has been to understand how to best synchronize the tricky triad
of customer engagement, IT culture, and strategic vision to produce
the best IT services possible. In her non-work time, she finds
fulfillment by volunteering with the Boston Area Rape Crisis Center,
running, cooking and eating out, and traveling.
- Wednesday, November 13, 2013
Adam Moskowitz et al
- Wednesday, October 9, 2013
BackupPC it's not just for desktops
Six years ago I embarked on a search for a new backup system for work.
This talk will start by discussing the requirements and evaluation
process for the backup system. This will be followed by a tour of
BackupPC and a discussion of how BackupPC is used to: backup multi TB
live databases, handle off site backups, handle bandwidth limitation
requirements for backing up sites scattered across the internet, how
compression and file de-duplication works to improve backup capacity
(fitting 52T of source data into 3.7T of backup space). The talk will
close with methods used for testing the integrity of backups and
John Rouillard's first computer experience was on PDP 11/44 running
BSD unix in 1977. After a couple of years of programming, he graduated
to system administration. He has authored papers on configuration
management tools, software management and real time log analysis. His
current interest is improving system administration processes by
incorporating best practices and standard skills from other
disciplines to solve issues in system administration.
- Wednesday, September 11, 2013
Is the MooseFS distributed filesystem in your future?
Peter aNeutrino (LizardFS.org)
This talk will focus on features of MooseFS which make it very
interesting choice for storage alternatives. MooseFS is used all over
the world to store hundreds of petabytes of production data. Starting
with installation and configuration the talk will move on to
management of MooseFS. Then the reasons for creating the LizardFS fork
of MooseFS will be presented. Finally the challenges to be solved to
make the LizardFS an enterprise ready storage solution will be discussed.
Peter aNeutrino started working in Poland 20 years ago writing
assembler for Demoscene. He sharpened his Linux skills working for
Siemens, Google and Gemius in Poland. He has spent his last 3 years
working on MooseFS. He now is building a team of developers to improve
MooseFS using modern development methods. This new Open Source release
is LizardFS that he hopes will change everything in today's storage
- Wednesday, August 14, 2013
No meeting - summer vacation
- Wednesday, July 10, 2013
Cloud storage options and DFS
Jeff Darcy (Red Hat)
This talk will be compare some of the cloud-storage options that are
out there, with a particular focus on distributed filesystems. GlusterFS and
Ceph will be compared in detail. Other options including HDFS, object stores,
and NoSQL document stores will also be discussed.
Jeff Darcy has been working on distributed storage since DECnet and
NFS version 2 in the early 1990s. Later he was one of the original
developers for MPFS while at EMC, and is currently an architect for
GlusterFS at Red Hat.
- Wednesday, June 12, 2013
Continuous Integration of Infrastructure
Stability is the name of the game for systems and networks operators. It's
what we expect and what our end-users expect, but it can be incredibly
difficult to achieve - a constant stream of security patches, dead-end
releases, broken software, broken hardware, and demands from other
internal stakeholders force change. Constant, never ending change.
Complicating this is the fact that these changes are rarely isolated. A
change on one system (particularly a lower-layer system like a router) can
ripple through entire services. We have, in effect, an integration
problem with every software update or configuration change we apply. So
to solve this, we look to the software development world to see how they
deal with it, and the answer is continuous testing and continuous
At TERC, we're currently combining configuration management (puppet),
hierarchical configs (hiera), monitoring (nagios), test
software (rspec/cucumber), and a software development CI
stack (jenkins/rake/vagrant) in order to address this. This talk will
focus on my experiences in rolling all of this out, what the limitations
of current software are, and touch on what my plans are to improve it.
Wednesday, April 10, 2013
- Wednesday, May 8, 2013
Asterisk and VOIP
The near-universal provision of voice services and their terminals
(called "telephones") predates the Internet. While on some level,
voice traffic via TCP/IP is just another protocol, there are
challenges in making it "just work" like the traditional phones that
we are all used to. There are the technical issues of the nature of
the data, interfacing with the still robust telephone network, and of
course the UI expectations and experience.
That means that the protocols involved - SIP and the related suite -
were developed in the setting of a preexisting, mature, and complex
switched network. I found that from the perspective of a systems
administrator or network engineer there are complications,
terminology, and conventions that aren't necessarily obvious.
This talk will provide insight into the these technologies from that
perspective to allow you to grasp the protocols and the context in
which they interoperate, using an example implementation of Asterisk.
Matt Simmons et al.
- Wednesday, March 13, 2013
Mastering Human Communication Patterns for Techies
Missed human connections in the software industry account for most of
our project failures. Improving communication can dramatically improve
individual and team performance. Typical frustrations:
- They just don't get it
- They talked at you for an hour and made no sense
- That meeting was like a construction committee for the Tower of Babel
In the same way that programming languages have interfaces and design
patterns, so do human beings. Problems that appear technical in nature
can usually be traced back to failed interactions between
people. We'll explore:
- The human interface between system administrators, customers, and managers
- Key patterns of communication including negotiation and resolving miscommunication
- Examine techniques for how to listen to and understand others
- How to be heard and understood yourself.
Mr. Hermes, principal consultant of Lexicon Systems, has over twenty
five years experience as a software management consultant, .NET
architect and developer. From start-ups to blue chips, Mr. Hermes has
served dozens of software companies striving to develop successful,
lasting enterprise systems. He has taught software architecture and
development at Northeastern University, Microsoft User Groups, and
Microsoft Certification classes at corporate training facilities. Cited on
National Public Radio, Forbes, and Reuters, Mr. Hermes has had articles
published by Media-N and MIT Press. He has served on the board of the
Institute of Management Consultants New England Chapter and is currently
director of Art Technology New England(ATNE).
- Wednesday, February 13, 2013
Centrify From a Sysadmin Perspective
"Centrify" is a commercial product that facilitates Unix and Linux
machines joining Microsoft Active Directory and using that for
authentication and directory. "directory", in this context, means the
information that was traditionally culled from NIS, NIS+ or LDAP, such
as automount maps. This talk will take a quick look at the problem,
give an overview of what Centrify does and how it does it, and offer a
few cautionary tales about implementation, based on the presenter's
- Wednesday, January 9, 2013
Adam Moskowitz et al.
A review of the events of LISA as seen by LISA attendees. Come and
share your experiences at LISA 2012.
- Wednesday, December 12, 2012
- Wednesday, November 14, 2012
Internet buffer bloat
Bufferbloat: Problem, Migitation, and Solution
VOIP and teleconferencing often perform much more poorly on today's
Internet than the Internet of a decade ago, despite great gains in
bandwidth. Lots of fiber, cheap memory, smart hardware, variability of
wireless goodput, changes in web browser behaviour, changes in TCP
implementations, and a focus on benchmarking Internet performance
solely by bandwidth, and engineer's natural reluctance to drop packets
have conspired to encourage papering over problems by adding buffers;
each of which may introduce latency when filled.
The mistaken quest to never drop packets has destroyed interactivity
under load, and often results in actual higher packet loss, as TCP's
congestion avoidance algorithms have been defeated by these
buffers. The lessons of the "RED manifesto" of 1997 have been
forgotten or never learned by a new generation of engineers.
Bufferbloat mitigation by tuning queue length is beginning to be
deployed in cable broadband systems. But solving bufferbloat requires
careful queue management that must be present anywhere a queue may
form. With the publication of the new CoDel AQM algorithm by Nichols
and Jacobson (and inclusion in Linux 3.5) we now have the opportunity
to solve rather than mitigate bufferbloat. I will also touch on
fq_codel, which combines stochastic fair queuing and CoDel, and why we
like the combination so much, and the remaining challenges.
Jim Gettys is at Alcatel-Lucent Bell Labs, USA. Today he works on
bufferbloat in all of its forms, including helping establish
bufferbloat.net to serve as a rallying point in
He was the Vice President of Software at the One Laptop per Child
project, one of the original developers of the X Window System, and
the editor of the HTTP/1.1 specification in the IETF. In 1997 he won
Bob Metcalfe's Internet Plumber of the Year award on behalf of the
group who worked on HTTP/1.1.
- Wednesday, October10, 2012
The State of ZFS
Peter Baer Galvin
ZFS has taken the world by storm, and is still advancing. This talk
will summarize the state of ZFS, including its availability, feature
set, and recent changes.
Peter Baer Galvin is a seasoned tech writer, columnist, consultant,
teacher and author. He is the CTO for systems integrator and VAR,
Corporate Technologies (www.cptech.com). He's a Lecturer at Boston
University and co-author of the Operating Systems Concepts
textbooks. He's given talks and tutorials at USENIX conferences and
- Wednesday, September 12, 2012
Solid State Drives: Use, Performance, Caching, and More
Dan Noé, VeloBit
Solid State Drives can deliver high performance, but their prices
still break the budget. SSD caching is a lower-cost method to improve
application performance by taking advantage of fewer Solid State
Drives to improve I/O. Dan will discuss SSD performance
characteristics, best practices and risks of SSD deployment, as well
as how SSD caching works and whether it can improve your performance.
Dan Noé is a Senior Software Engineer at VeloBit; previously,
Dan was an engineer at IBM/Netezza,where he worked on database storage
layer technology for the massively parallel Netezza Database
Appliance. Dan holds a B.S. in Computer Science from University of New
Hampshire, is an avid pilot and maintains Linux servers in his spare
- Wednesday, August 8, 2012
- Wednesday, July 11, 2012
Solaris Dynamic Tracing - DTrace
DTrace is a revolutionary software framework that enables
unprecedented observability up and down the entire software
stack. DTrace was first introduced in Solaris 10, and continues to
ship with Solaris, with ports to Mac OS X and FreeBSD. Others are
underway. This talk will provide an overview of the DTrace framework
and key components, as well as a tour of using DTrace to measure and
observe system behavior.
Jim Mauro is a Principal Software Engineer for Oracle
Corporation. Jim's focus for the last several years has been systems
performance, doing both internal performance-related engineering
projects, as well as engaging in real customer production performance
issues. Jim's most recent work involved performance and benchmarking
of Oracle's ZFS Storage Appliance. Jim is the co-author of Solaris
Internals (1st and 2nd Ed), Solaris Performance and Tools, and
recently published DTrace.
- Wednesday, June 13, 2012
Keeping up with Systems Management across Windows, Linux and Mac platforms.
How do you keep all of your systems (Windows, Mac, Linux)? Do you find
yourself having to use multiple toolsets? Enterprises keep adding new
types of system s (a resurgence in the Mac community and new Linux
applications being developed every day) that only increase the demands
on the systems administration team. On top of that, there are growing
regulatory and compliance (software licensing) demands placed on the
team. How can you streamline the systems management processes
(software distribution, asset management, patching, O/S deployment,
compliance reporting, etc.)? The Dell Kace systems management
appliance was designed to make it easy for you to manage all of your
systems from one console. We'll discuss common challenges that systems
administrators face today and how best to address them.
Harold Moore has worked in the systems management field for over 15
years. Harold worked at Novell from 2000-2007. He worked on the System
Engineering team that was responsible for systems management and the
Suse Linux/Open Enterprise Server. Harold then joined Altiris/Symantec
working in systems management group supporting management of MAC,
Linux and Windows. Harold Joined Dell in March 2011 and works in the
Dell-Kace division. Harold has a BA from the University at Albany in
Communications and Computer Science. He also hold a MS in Computer
Science from Long Island University, CW Post.
- Wednesday, May 9, 2012
What does a CIO do anyway?
The average job life of a CIO is about 2 1/2 years, it even comes with
the pleasant acronym of '*C*areer *I*s *O*ver'. The Broad Institute
just hired their first CIO, and will try to justify his existence at
this presentation and discussion.
Martin Leach is chief information officer at the Broad Institute He
came to the Broad from Merck & Co., where he led IT for Discovery and
Pre-Clinical Sciences across all the North American research
sites. Over his career he has provided support and strategic vision
for IT, informatics, and data-mining activities at a range of life
sciences organizations. Martin received his B.Sc. in cell and
molecular sciences from Anglia Polytechnic University and his Ph.D. in
pharmacology from Boston University School of Medicine.
- Wednesday, April 11, 2012
A fresh look at SELinux and what it is complaining about.
The Four main causes of SELinux problems.
- Labeling Problems
- SELinux has to know how you configured your processes
- Bug in Policy or an Application.
- Your machine has been compromised.
Daniel Walsh has worked in the computer security field for almost 30
years. Dan joined Red Hat in August 2001. He has led the SELinux
project, concentrating on the application space and policy
development. Dan helped developed sVirt, Secure Vitrualization. He
also created the SELinux Sandbox, the Xguest user and the Secure
Kiosk. Previously, Dan worked Netect/Bindview's HackerShield and
BVControl for Unix, Vulnerability Assessment Products. Dan worked for
Digital Equipment Corporation on the Athena Project along with
designing and developing the AltaVista Firewall and AltaVista Tunnel
(VPN) Products. Dan has a BA in Mathematics from the College of the
Holy Cross and a MS in Computer Science from Worcester Polytechnic
- Wednesday, March 14, 2012
Bacula: An Introduction to an Open Source backup system
K. M. Peterson
What should be a simple problem to solve - having a copy of important
data in case of hardware error or human mistake - turns out to have
complex (and expens ive) solutions. Bacula is an open-source
application that runs in most popular environments, supports disk and
tape-based backups, and utilizes a database for managing its catalog
of file versions and backup media. This presentation will discuss
Bacula's functionality, including its features and some implementation
details, and provide a short example of a working configuration.
K. M. Peterson has worked as a manager, systems administrator, and
consultant in academic, commercial, and non-profit environments. He's
interested in topics in data management, networking, security and
automation. Currently, he is seeking the next challenging role, and
digging deeper into interesting technologies he has encountered over
the past several years.
- Wednesday, February 8, 2012
An informal overview of the oVirt project: status, goals and a brief demonstration.
Dave Allan worn a number of hats: sysadmin, operations manager, field support staff, QA engineer, software developer. I'm currently a software development manager at Red Hat where I am the PHB of the libvirt team and interact freqently with the oVirt team.
- Wednesday, January 11, 2012
- Wednesday, December 14, 2011
- Wednesday, November 9, 2011
Tracking issues - experiences from the field
Christopher Allison, Tom Bechard, John Rouillard, Tony Rudie, Clarence Smith
- Wednesday, October 12, 2011
Cloud Filesystem HekaFS
Modern open-source distributed filesystems make it possible to provide
file services at a scale and level of availability that's finally
competitive with proprietary options. What they don't do - yet - is
enable secure sharing of those resources between multiple user bases
or organizations who pay for them. This talk will focus on how
GlusterFS works to solve the first set of problems, and how HekaFS -
which is based on GlusterFS - is solving the second. If you're tired
of having to deal with umpteen departmental file servers, each
configured differently, this approach might provide some relief.
Jeff Darcy has been working with network, cluster, and distributed
filesystems for about twenty years - since DECnet was still relevant
and NFSv2 was new. Since then he has gained scars from EMC's MPFS (for
which he was one of the initial developers), Lustre, and GlusterFS. He
is currently at Red Hat, where he's the project lead for HekaFS and
all-around "cloud storage" expert.
- Wednesday, September 14, 2011
Converged Networks, Voice / Video / Storage / Data
How do you keep everything running smoothly while giving technologies
that require low or constant latency what they need and still be able
to watch that dog skateboarding on YouTube. Will discuss how Quality
of Service (QoS) works on both Layer 2 and Layer 3 devices and why you
probably need both. How should traffic be tagged and retagged with
policy decisions. If time permits we may even have some time to go
into why net neutrality isn't as cut and dry as you may think.
Technical details will be based off of Cisco devices but most concepts
will translate to any modern equipment.
Our speaker, Ryan Sutton, is a Systems Engineer at a local Gold
Certified Cisco Partner. Ryan specializes in large scale Routing and
Switching, Voice, and Data Center designs and implementations. His
solutions often include interworking between multiple networking
vendors and technologies. Current projects include sub-second core
convergence, and multi-city/multi-vendor VoIP installations.
- Wed, July 13, 2011
Ruby: More Batteries, Fewer Brackets
Aaron D. Ball
Ruby may be most familiar as the language behind the Rails web
framework, and Perl as the "Swiss Army chainsaw" that no sysadmin can
live without, but they have a lot more in common than you might
think. Ruby comes out of the box with a great set of sysadmin tools,
from text processing to Unix system interfaces to TCP servers, and has
a syntax about as terse as Perl but with object-oriented and
functional-programming idioms that make your code easier to write and
understand. Whether you're new to scripting or you've been typing line
noise since 1987, this talk will show you another way.
- Wed, June 8, 2011
Robert Thau from Smartleaf will present their Tuttle system configuration tool.
- Wed, May 11, 2011
Automating Inventory, Deployment and Configuration of Your Windows Infrastructure
Like most IT professionals, you are an administrator in a
heterogeneous environment. You have a myriad of tools to inventory,
deploy and configure your Unix/Linux machines but how do you do this
for the rest of your machines? Come to this session to learn about the
tools you must have in your toolbox to inventory, deploy, and remotely
configure your windows desktops and servers. We will discuss the free
tools as well as the top of the line fully automatable solutions
available by Microsoft.
Dan Stolts is a technology specialist with more than 24 years in the
industry. He is proficient with many Microsoft products especially
those in the server area and holds many certifications including MCT,
MCITP, MCSE, TS, etc. Dan is currently specializing in Systems
Management and Security and is also very passionate about
virtualization technologies. Dan is and has been a very active member
of the community. He is the current president of Boston User Groups.
- Wed, Apr 13, 2011
The Path to Senior Sysadmin
Being a senior system administrator is about more than knowing all the
options to mount(8) or that modprobe is what's used to replace that
buggy kernel module with the latest version. Rather, a good senior
sysadmin will have a wide knowledge of relevant technical topics,
in-depth knowledge of one or more technologies, good interpersonal
skills, and the ability to manage "problem users" and will be
comfortable making presentations to and negotiating with mid- and
upper-level management. This talk will cover the skills a senior
sysadmin needs and why they are necessary and will provide some
suggestions for how to acquire these skills.
For nearly one-third of his sysadmin career, Adam Moskowitz held
titles such as Senior System Administrator, System Architect, and IT
Manager. Despite having returned to his roots as a programmer, Adam
remains active in the sysadmin community, including running the LISA
Advanced Topics Workshop and serving on the LOPSA Leadership
Committee. He claims he does all of this only to support his hobby
Advanced Topics Workshop and serving on the LOPSA Leadership
Committee. He claims he does all of this only to support his hobby of
judging barbecue contests and to keep food in his puppy's bowl.
- Wed, March 9, 2011
How Splunk manages our Junk
Jim Donn and Tim Hartmann
As environments grow and systems become more complex, building and
managing a usable centralized logging infrastructure can be a daunting
task. In this talk, we will walk through our real-life experiences
implementing Splunk as our centrali zed logging infrastructure for our
Network, Systems, Security, and Application teams. Over the past three
years, we have had to change our strategies and architecture to
account for organic customer growth, changes in team requirements, and
evolutions in technology.
Jim Donn, Harvard University Network Services Group (UNSG)
Senior Network Management Engineer
Tim Hartmann, Harvard University Network Services Group (UNSG)
Senior Systems Administrator
- Wed, February 9, 2010
Project Caua: Private Sector, Environmentally Friendly Jobs with Free Software
Project Caua is an Open project to create millions of private sector,
environmentally friendly jobs utilizing FOSS in urban areas of Latin
America, and millions more around the world. In addition, Project Caua
will open an avenue for free (as in beer) wireless Internet to help
defeat the digital divide, and to provide low-cost training to move
people off unemployment and create taxpayers. The specifications for
Project Caua can be found
at http://www.projectcaua.org/. This
talk will outline Project Caua and open the discussion for actual
Jon Hall is the Executive Director of Linux International
(www.li.org), an association of
computer users who wish to support and promote the Linux Operating
System. During his career in commercial computing w hich started in
1969, Jon has been a programmer, systems designer, systems
administrator, product manager, technical marketing manager, author
and educator. He currently works as an independent consultant, and is
currently involved with bringing environmentally friendly computing to
- Wed, December 8, 2010
Usenix LISA 2010 conference recap.
- Wed, October 13, 2010
Using MySQLtuner 2.0 to monitor and improve mysql performance
Sheeri K. Cabral
With help from Major Hayden, mysqltuner's original author, Sheeri
K. Cabral of the Pythian Group has modified mysqltuner to be more
comprehensive, to output information and to have a "spreadsheet" mode
where the results of mysqltuner are outputted as a single column, so
that you can easily compare subsequent runs of the modified
mysqltuner -- for example, running it monthly or weekly to see how
performance is progressing (or degrading). There is also a truly
offline mode that requires no database connectivity where files
containing the output of SHOW GLOBAL STATUS and SHOW GLOBAL VARIABLES
are used. This talk will go through how the the modified mysqltuner
works including how easy it is to change what is checked and the
thresholds, so that you can easily do one-off sanity checks as well as
Sheeri K. Cabral (The Pythian Group) is a noted MySQL community activist who recently wrote The MySQL Administrator's Bible.
Keep up with her MySQL writings at http://www.pythian.com/news/author/sheeri/.
PDF Slides and Openoffice slides.
- Wed, September 8, 2010
Built on a modular, yet integrated architecture, WhatsUp Gold is an
affordable and easy-to-use solution that scales with the size and
complexity of any physical or virtual IT infrastructure. From a single
console, WhatsUp Gold supports standard IT management tasks including
automated discovery, mapping, real-time monitoring, alerting,
troubleshooting and reporting. Rich Makris will walk through the
benefits of using WhatsUp Gold and how it can make your life easier.
As a Sales Engineer for the Network Management division, Rich's focus
is on helping customers solve their IT Management needs with WhatsUp
Gold and Event Log Management products. He has held various systems
and network positions for more than 15 years in government,
manufacturing, financial services, and at service providers. Rich also
holds certifications from Cisco, Microsoft, and Novell.
- Wed, May 12, 2010
Daniel has been using IPv6 for fun (and to get things done) for quite some time. Come learn what has and has not worked for him. Details of his personal dual-stack IPv6 setup will be presented as well as other experiences.
- Wed, 14 Apr 2010
"Building 16 systems in 16 minutes with xCAT"
Ali Tayarani will discuss how we use xCAT to manage several hundred hosts in our general-purpose LSF-based compute cluster. Slides
"Redefining Compute Nodes and Provisioning"
John Hanks will discuss his grand vision for the future (stateless compute nodes managed with Perceus), exemplified by our new genome-sequencing GridEngine cluster.
- Wed, 10 Mar 2010
"How to Interview a System Administrator"
This will be a shortened version of Adam's LISA tutorial.
The full description can be on the LISA 2007 web site.
- Wed, 10 Feb 2010
- Wed, 13 Jan 2010
“I Got My Jet Pack and I'm Still Not Happy”
- Wed, 9 Dec 2009
Thirty Minute Tools
John Rouillard (and others)
- Wed, 11 Nov 2009
- Wed, 14 Oct 2009
(was there a meeting?)
- Wed, 09 Sep 2009
“Log Analysis with the Simple Event Correlator”
John P. Rouillard
- Wed, 12 Aug 2009
Everything I Know About Sysadmin I Learned
in the Back of an Ambulance
John P. Rouillard