Category Archives: Linux

The O/S that keeps me free (as in speech and beer)

Fedora summer 2016 internships.

There are three USA internship positions open at Red Hat on the Fedora Engineering team. These internships are all available this coming summer (2016). Read below for some more details.

I feel like my recent blog posts have all been about openings. But it’s nice to be able to offer them. Each internship on our job site is linked in the bullets below, so you can apply for any for which you feel qualified.

  • Software engineering internship – This internship focuses primarily on Python software development. You’ll take designs and build them into widgets for our Fedora Hubs project. You’ll collaborate with our applications team on back-end technology for the community. You’ll also work with our infrastructure team on production deployment. This internship also includes other related projects, not limited to just Hubs. The intern is preferably in Westford, MA for the summer. But that’s not strictly required for a well-qualified candidate with experience working in a global open source project. (In other words, if you can succeed at keeping touch daily with the team as a remotee, don’t let the location throw you.)
  • User experience and design internship – This intern will work directly with our designers Máirín Duffy and Ryan Lerch on research and design for Fedora projects such as Hubs. You’ll do user research and work on improving interaction in our web apps and other associated projects. You’ll collaborate with the other interns, our whole development team, and the Fedora community at large. This job is in Westford, MA.
  • Web development internship – This intern will work primarily (but not exclusively) on web apps at the front end. You’ll turn design into reality, and streamline existing applications to designer specs to build a more unified project. As with our other internships, you’ll be able to explore and work on other projects too. You’ll work with our design team, the community Websites team and our application developers to create and improve web tools our contributors will use for years. This job is preferably in Westford, MA for the summer. But we’ll consider remote for an exceptionally qualified candidate.

These jobs should end up being quite competitive. So I encourage you to get your application in as soon as you can!

Last year, we were fortunate to have superb interns who produced fantastic results. Meghan Richardson worked on many of the initial designs for Hubs in its early stages, interviewing potential users and then creating the workflows and feel for this new tool. Nate Yazdani worked on statscache, which compiles data from our messaging bus to make it available quickly to other apps, without constantly crushing our data store.

Red Hat is a fantastic company at which to intern, and we pride ourselves on an open and inclusive culture. These are not “fetch coffee” jobs — you’ll work on real problems alongside real experts.

We can’t wait to see what you can do!

Outreachy internship available

Here’s a fantastic opportunity in open source for folks from underrepresented backgrounds. The Fedora Engineering team has an Outreachy internship available December 2015 to March 2016. Applications are being accepted at the Outreachy site. The deadline is 1900 UTC Monday, November 2, 2015.


Are you familiar with Outreachy? It’s a program designed to boost participation in open source by underrepresented groups. My employer, Red Hat, is sponsoring this internship.  The Fedora Project, which my team at Red Hat supports, is proud to partner in offering it. This Fedora internship is an opportunity to work directly with our senior software engineers on a major project called Fedora Hubs.

Hubs is an initiative to bring better and more timely information to Fedora’s large community of participants and contributors. Several of our engineers will collaborate with you throughout the internship period on this project. We’ll use Hubs as a foundation for regular, daily mentoring of our Outreachy intern. You’ll use best of breed technologies like Python, Bootstrap, jinja templates, and more. Best of all, you’ll work alongside the team in real time.

You can read more about the internship and qualifications here on our wiki. The page also has general information about Fedora, such as our mission and how we work together.

But this internship isn’t just extending your skills, or working with awesome people. (Even though that’s a great incentive!) It’s also about bringing the open source way to thousands of people. This project lets contributors build sub-communities, or hubs, around their interests. “Standing on the shoulders of giants” is one of the greatest benefits of open source. The same goes for using open source to build communities. It’s a natural fit and a wonderful outcome for a successful internship.

Are you interested in diving deeper into Hubs, to see what this internship is about? Check out this recent blog post from our own Máirín Duffy on the purpose, design, and mockups of the Hubs project.

We’re looking forward to seeing your application. Remember the deadline is 7pm in UTC time on Monday, November 2, which is mid-day for any USA applicants.

Flock 2015 thoughts.

Getting started at Flock

Like everyone on the Fedora Engineering team, I was in Rochester for the Flock conference last week. After several flight delays on our direct flight from DCA to Rochester, Justin Forbes, Ricky Elrod, and I finally arrived a little after 9:00pm — about four hours late. Thankfully Josh Boyer came to pick us up at the airport.

Flock had a team of organizers within OSAS (and Josh also assisted throughout). As a former FUDCon organizer, though, I know the value of extra hands showing up to do work. Since old habits die hard, I showed up expecting to help out behind the scenes. That means I didn’t get to see a huge amount of content I was personally interested in. But in return hopefully everyone had a smoother Flock experience, especially speakers.

When I arrived, I reported to Tom Callaway, Ruth Suehle, and Josh. They got the conference rooms opened, and I helped set up the speaker workstations. We worked pretty late, well after midnight. Things were looking a little bleak at that point, with execrable network bandwidth, no projectors, no screens, and no audio for the ballroom.

Fears and worries abate

Nevertheless, the next morning Josh and I got up early and grabbed coffee at nearby Tedward’s. This place was a godsend, although their 7:00am opening time forced us to walk around a bit until we could get in.

We went down to do some additional setup. The organizers had worked with Remy DeCausemaker to get a bunch of loaner projectors from RIT so we’d be ready for the first sessions at 10:00am. (EDIT: According to Remy, Tim Duffy and Dan Schneiderman are the heroes of this particular day; see comments below.) So at least our speakers would be in OK shape. I helped Josh and Tom get everything ready in those rooms, while Ruth made sure registration and other logistics were under control. I missed Matthew Miller’s keynote, but I’d seen at least some of the material previously.

After lunchtime, things continued to drastically improve. The rental projectors showed up, along with small screens for each room and big speakers for the ballroom. The wireless internet improved quite a bit when a switch flip occurred due to our conference starting up. (It was dismal Tuesday night!) We had all the speakers trained on how to record their talks locally, to get around the constrained network bandwidth.

Suddenly things were looking up! Not surprisingly, the Fedora Engineering team dinner that night at The Old Toad was much more enjoyable. Since I wasn’t overly worried about the conference experience for the speakers and attendees any longer, it was easier to relax and enjoy the company of the team. I was so happy that we were able to get together in one place, since we really only get to do that once a year. (Incidentally, our friend Stephen Smoogen was absent from Flock due to family commitments — we missed you, Smooge!)

Fedora contributors at Flock gather at Victoire for dinner
Fedora contributors at Flock gather at Victoire for dinner

I continued to monitor speaker rooms most of Wednesday and Thursday. I managed to make it to a couple sessions where I wasn’t sure there would be any senior Fedora leadership around. For example, I attended the Fedora Magazine session by Chris Roberts as well as most of the Fedora Hubs session by Máirín Duffy and Meghan Richardson.

I attended and loved Major Hayden‘s (of Rackspace fame) Thursday keynote on fighting impostor syndrome. It was one of the most practical that I’ve seen on this topic. I feel impostor syndrome is just a fancy way to refer to insecurity, a common trait for conscientious people. But that doesn’t make the strategies Major outlined any less useful or thoughtful. He gave a great talk — engaging and humorous without diluting the material. If you have a chance to invite him to a conference to speak, definitely do so!

I gave my own talk on Remote Ninjutsu on Thursday afternoon. The slides for the talk are here, although the video will be more useful for context. All the Flock 2015 videos are supposed to be available at some point in the next couple of weeks. Stay tuned for announcements about them.

The Thursday night social event at the Strong National Museum of Play was fantastic. It was a great way to blow off steam and enjoy the company of fellow Fedorans. I’m not sure how the organizers managed to find such a perfect venue!

Workshops and Flock wrap-up

On Friday I enjoyed the keynote by Jon Schull of eNable, the community that is flipping the script on prosthetics provision through 3D printing. It was a very moving look at how people are applying open source to make the world better for people in need.

Then the workshops beckoned. Now that I’d finished my Flock duties helping speakers and attendees, I was able to attend several sessions that were relevant to me personally, including the Fedora/CentOS rel-eng joint session, and my own on revamping the Flock software stack.

Once again, the Friday night social event at the George Eastman House was marvelous. It was a beautiful, grand mansion and the tour was quite interesting. I’d love to go back there sometime to see the exhibits I missed!

The music parlor in the George Eastman House
The music parlor in the George Eastman House

Flock conferences are always especially great for their hallway track. So many discussions can be had or progressed that way with high bandwidth. The challenge is always to move that discussion to a transparent context if it involves people not present, though. I’ve been seeing many trip reports from people’s blogs about Flock, and resulting list discussions, so I think that process is well underway.

Of course, that means Flock is a very engaging event. It takes a lot of attention and brainpower to shift focus for all those conversations! As a result, by Saturday afternoon I know I was fairly exhausted — in a good way, though. Several other people I know felt likewise, and commented on how well the conference had gone. In fact, I heard a number of comments that this was the best Flock, and even Fedora premier event, yet. The OSAS folks deserve special recognition for pulling off a fantastic conference.

Sunday started with a couple meetings, including with Matthew Miller and Jan Ku?ík, our new Fedora program manager. Then, after seeing a few other friends and colleagues off, I got to the airport. I relaxed in a lounge over beers with Kevin Fenzi, Jan Zeleny, and Stephen Tweedie, before we went to our respective flights. Then after a quick flight home, it was the usual “fun” making my way down I-95 from the airport to home. Monday morning was right around the corner…

Here’s to another great Flock, and to doing it again next year!

Flock attendees wind down after the conference ends... with more hacking!
Flock attendees wind down after the conference ends… with more hacking!

Fedora Engineering team opening.

The team I manage at Red Hat, the Fedora Engineering team, includes people who work on Fedora system administration, release tooling, application development, and design. We have a job opening for an engineer to work with our infrastructure applications team on some challenging, fun, and forward-looking problems:

  • Building and enhancing our tools for producing and shipping cloud images for a variety of providers (experience definitely required!)
  • Working with the Fedora Cloud SIG and other community members to resolve issues related to the Fedora Cloud edition and associated tools and processes
  • Collaborating with the rest of the Fedora team to automate, automate, automate All the Cloud Things
  • Working with the rest of the team on non-Cloud related projects too, such as Fedora Hubs
  • Staying abreast of and aligned with work going on throughout Red Hat related to Fedora and cloud technology
  • As with all Fedora Engineering jobs, communicating openly and continually with the whole community, and building community around everything you do using open source best practices

Our team uses a lot of Python. We create code upstream that is widely consumable beyond just Fedora, and we deploy our work on both Red Hat Enterprise Linux and Fedora. We do that work openly: collaboration via git repositories, rapid and constant communication via IRC, frequent discussion through our mailing lists, and taking opportunities to gather and build community around our work. Simply put, we love open.

Although the description says the job is in Westford MA, USA, in reality we’re a highly distributed team. While this job is originally conceived as an entry- or journeyman-level engineer in the Westford (or possibly Raleigh NC) Red Hat office, we’re also open to experienced remotees outside the USA. The right candidate is a team player, fully engaged and passionately committed to delivering results with their colleagues, wherever they might be.

Does this sound interesting to you? Go read the full description of the job, and then apply online.

PulseAudio is still awesome.

PulseAudio is an indispensable part of my Fedora Workstation. While PulseAudio may have detractors, my experience has been all good. Most sound problems I’ve seen come down to one of the following:

  • Bad drivers at a lower layer (not PulseAudio’s problem)
  • Bad hackery in the distro (ohai *buntu)
  • People parroting years-old Internet “wisdom” (does that happen?)

But in a well-integrated system, PulseAudio is awesome. That’s definitely the case with Fedora Workstation. That includes Fedora 22, which is what I’m using now.


I’m attending a number of conference calls this week. Lots of them. The last two days have been eight hours of meetings. For example, today I’m at a Fedora Hubs workshop with fellow community members. I usually use my laptop to attend these conference calls. I use it with a USB headset attached. But having a headset on for that long is uncomfortable.

I also have a desktop computer in my office hooked to a big speaker set. So I use PulseAudio to route sound from my conference calls to those speakers. Now I don’t have to wear a headset to listen in.

I do this using the paprefs and pavucontrol utilities. I find these utilities very useful for more audio control. I can watch and move streams, all though an easy, comfortable GUI.

My home office setup, courtesy of PulseAudio.
My home office setup, courtesy of PulseAudio.

Thanks for the memory, PulseAudio

But that’s not all!

I was in the conference call, but there were updates available on the desktop computer. I decided to see what happened when I rebooted to get the updates. (Since my laptop was on the call, it wasn’t a problem.) I restarted the desktop, and the sound automatically moved back to the laptop speakers! That was cool — I didn’t lose any of the conference.

But it got better. After updates finished and the system restarted, I logged in. The sound automatically went back to the desktop speaker system! I was really impressed.

PulseAudio can track where audio streams are going, and send them back to the original devices when restarted. I love this feature! I remember the early days of Linux audio. It seemed like every single operation was hit-or-miss. I’m glad I don’t have to mess around with the system at that level anymore. I can just concentrate on my tasks, and things Just Work(tm).

Flock bus from Boston/Westford.

Are you (1) coming to Flock 2015; (2) located, or planning to be, in the Boston, MA or Westford, MA area; and (3) interested in taking a free, power and wifi-equipped bus if offered?

If so, then pre-register by 1900 UTC Friday, June 5, and make sure you indicate in your registration you are located in the Boston/Westford area. I’ll collect the final number of these signups on Friday afternoon US/Eastern time. We’ll use that number to determine whether or not it’s economical to book the bus.

The announcement about the bus will be made next week, so there will be plenty of time to make appropriate travel plans either way.

Fedora under construction?

Fedora’s quality makes complacency easy. But in truth, we’re always under construction — or we should be. You could call that constant disruption by different names. Risk positive. Forward leaning. Embracing change. Since inception, Fedora was intended to avoid the status quo. So what’s next for shaking up said status?

Courtesy of US Library of Congress.

Season recap

As you’re probably aware, starting with Fedora 21 our releases are a bit different. We now have different editions, each serving a specific type of consumer. The editions today are Workstation, Cloud, and Server. The editions differ in mostly minor ways, though, and we build them mainly the same way. That’s why these editions aren’t the end goal of change. Rather, they’re a step in changing what Fedora releases.

Matthew Miller, Stephen Gallagher, and others have been steadily laying out a vision for change in Fedora. Admittedly, that vision’s high level. It uses words like “Rings” to simplify and amplify concepts that are about change. These concepts are well attuned to higher level Fedora goals. While editions can effectively broaden Fedora’s appeal to different consumer audiences, we also want to attract more makers of things.

These days, the largest maker audience is beyond just those who care about building a platform. As Red Hat’s Paul Cormier said last year, “The application is king.” Makers of applications and things above the OS are always on our minds. So how does this relate to the Fedora release of the future?

Season preview

For many years now, we’ve been building our release essentially the same way. We take a big bag of packages, like building blocks, and glue them together in a group that makes sense. We wrap that group in a shippable format (or several), sometimes with an installer, validate, and release it. Lately many Fedora folks are thinking about what that future release looks like, though. I would offer that the future release should be some combination of a strongly managed center, curated stacks, and an expanding nebula of containers.

Managed center? Does that mean the return of Fedora Core? No. Forget about Fedora Core. It’s dead, and it’s not coming back. Having the central part of the OS carefully managed in the community isn’t the same thing. Fortunately, there are emergent tools to do this. Among them, Project Atomic looks like one of the best bets in the space we care about. Atomic makes shipping an integrated, validated set of content easier. That content still comes from the packages we know well — kernel, glibc, bash, and others. But the rpm-ostree basis of Atomic can prevent slew in an installed system.

What do I mean by “slew”? Right now, the only Fedora release we know well is the one we put out at GA. After that, all bets are off. Any user potentially has a random subsets of updates on top. Saying “I’m using Fedora 22” is not very meaningful soon after release. That slew also makes validation, troubleshooting, diagnosis, and recovery unnecessarily hard. What if we manage this central content more carefully, using a model like rpm-ostree? We could validate a release more regularly, at least at a basic functional level.

There are other benefits as well. What if we constructed Rawhide using the rpm-ostree model? Imagine that an important core piece of the OS broke. We’d want to file that bug, track, fix, and update. Right now, hundreds of developers have to manually, sometimes painstakingly, fix their systems. This wastes large amounts of aggregate time. This problem is largely why many interested developers don’t run Rawhide as much as would be helpful. In the new model, if you’re invested in the problem, you can still work on it, of course. But the rest of the Rawhide users could, with a single command, back out to the previous tree and keep working. This keeps more interested (and interesting) people using Rawhide.

You could probably make a case for file system snapshots here. I think those are still useful for user data in this model. It’s not clear that snapshots would solve the slew problem above without imposing restrictions on them in some fashion. Would users be happy with that? Hard to say.

So what about these curated stacks? Well, to be honest it’s not yet clear what tech wins here. On one hand, enhancing rpm-ostree to allow layering might be a way forward.  Currently rpm-ostree is somewhat monolithic in that you can’t really mix or match stacks readily… yet. My understanding is the Atomic guys are thinking about this problem already, though, and how to solve it. So I expect the code will catch up to (and maybe overtake) the concept before we know it.

Alex Larsson is also doing interesting work on what he calls “sandboxed apps.” Sandboxing in this model might not be too different from containers. The concepts seem quite similar. Throw into this mix the recent progress on overlayfs, which is now part of the mainline Linux kernel. What you have is ripe ground for a new method to build and release big swaths of a platform.

Again, we have building blocks of a solution for interesting problems, some long-standing. The problems of the central core above are shared at common application layers as well. But it’s useful to detach them at key dividing lines for obvious reasons. In some way, shape, or form, it seems inevitable that Fedora must take a swing at these problems, even if it’s on a per-edition basis.

Courtesy of Glyn Lowe (CC-BY 2.0).

This leaves the nebula of containers I mentioned. A year and a half ago, containers were all the rage. People were thinking about how they affected the technology landscape for infrastructure. Perhaps some people reading this were thinking about containers as a fad. Time and consumers and commercial customers have pretty much proved that wrong. Containers allow app developers and users to move swiftly (or not) independent of the OS. The technology is here to stay, because it mostly makes OS maintenance issues someone else’s problem. In this case, ours — Fedora’s.

For a long time we’ve relied on people working the application layer to radically change their methods if they’re interested in deploying on Fedora. But frankly, time has shown the world doesn’t care to do that. So our choice is to adapt or face irrelevance. Matthew Miller has spoken to this point several times, so I won’t belabor it. My only other point is that, however we build the future Fedora release, it should make King App comfortable.


For those people who might ask why we should take on this work, I’d start with a couple of thoughts. First, what’s in it for me as a contributor? Well, it depends first on whether you care about an edition that might use this model. I’m pretty involved in Workstation edition. For some time we’ve been interested in how to update Fedora for consumers at the OS and application level, rather than packages.  And something like a core + containers model using Atomic directly solves that problem. The Cloud SIG already has an Atomic image designed primarily for container management. Their user base has different expectations from Workstation. But clearly there’s room for great collaboration here, and I expect for Server too.

Another reason this is interesting is app vendor involvement in Fedora. Containers abstract the distribution problems away from application vendors. We know Fedora is in a lot of embedded hardware and other projects in the Real World. That problem space fits well with our current platform construction model. As I said before, we don’t necessarily need to stop doing that. But at the same time, embracing the rise of the app container lets us more effectively reach the developer audience. This is not just about our Workstation edition but, more generally, people who build and make interesting things. This is also somewhat tied up with the implementation of Rings. That’s why I look forward to Matthew and Stephen driving more detail and progress on that front.

Finally, by solving this problem we can effectively influence the value of future RHEL, as well as CentOS and our other family members. That value is one reason Red Hat invests time and resources in our community. Making changes to grow that value is always a win-win for Fedora as well.

The hero’s journey

So, does all this mean we won’t have live or installation ISOs in the future? Not necessarily. They could still be useful. It would depend, as most things do, on what it takes to validate and maintain all those things in the long run. For example, I don’t see this idea necessarily affecting spins. Communities around those releases are accustomed to how they build their bits.  But I think at least one (maybe more) of the official Fedora release editions will need to opt for this new model if we’re to make progress.

The entire set of changes needed isn’t yet clear, of course. One thing that is clear: release engineering process is, by definition, central. And there’s no doubt we’re looking at a healthy chunk of work. But I believe strongly enough in the possibilities that our team has an extra full-time person now to drive planning of those changes, consult with the Fedora release engineering team, and build a community around the tools needed. We are going to emphasize modularity, so the winning technology of tomorrow can plug in to releases down the road. The initial goal I’d like to set is to prototype a release of at least one Fedora edition for F23, and be part of F24 official release at GA.

That means it’s going to be an exciting year of construction ahead for Fedora. So please excuse our dust!

Courtesy of Jakob Montrasio (CC-BY).


Another new Fedora team member.

I’m excited to tell you about another* new addition to the Fedora Engineering team. In mid-April, Laura Abbott joins our kernel team. She comes to us most recently from Qualcomm. Her projects there included memory management and debugging support. She’s also a regular contributor to the Linux kernel upstream. Laura will work with Josh Boyer and Justin Forbes on projects relating to both the upstream and Fedora kernel.  We’re happy to have Laura joining our team soon. Please help us extend her a warm welcome.

* As you know from my last announcement, Adam Miller also joins us in April. He’ll work on release tools and process for Project Atomic and other leading edge technologies.

New addition.

I’m proud and pleased to announce that Adam Miller is joining the Fedora Engineering team in mid-April, as our new release infrastructure guru. Adam is not new to our community — he’s been involved in Fedora for a number of years. He also has a long history of working on automation, release, and continuous integration in OpenShift, and I’m thrilled he’ll be bringing his experience and energy to our team. Among other things, Adam will be working on the next generation of release tools and deliverables in Fedora, in conjunction with the release engineering team and our community. I hope the Fedora community will join me in welcoming him.

New features in Fedora 22 Workstation.

Matthias Clasen recently posted some updates on the Fedora development list about new features in Fedora 22 Workstation. As you may know, we’re getting ready to issue an Alpha, so it’s a great time to try out these changes.

  • Notifications have improved significantly in Fedora 22. The message tray no longer hides in the bottom, but is subsumed into the top bar. You can review them from the calendar popup. (Here are the original mockups.) Please try out the notifications with your favorite applications and report bugs if you find functional problems.
  • The GNOME Shell theme has been refreshed.
  • The login screen in Fedora 22 Workstation now runs over Wayland, but falls back to the standard Xorg server if something goes wrong. This is part of Fedora’s overall strategy of getting things running on Wayland early. Wayland as a session option was available in Fedora 21, but we are ultimately moving toward having it as the default in a future release. Note that the fallback calls Xorg in a slightly different way, so you may see bugs (and filing a bug report is welcome).
  • The UI for installation of codecs, fonts, MIME types, and so on is now handled by GNOME Software. This change further enhances the sense of GNOME Software as a one-stop shop for installing things, which was a highly praised feature in Fedora 21.
  • Nautilus has had multiple improvements, including more pleasant and logical context menus and controls (here are the designs).

Fedora 22 Alpha is scheduled to release on March 10. We hope you’ll give it a try, so you can test drive these new features. Those of you knowledgeable about Fedora 22 pre-releases can find early test candidate images for Alpha in the usual places. Please use Bugzilla to file helpful bug reports. If you need help filing a bug report, this wiki page may be useful.