Using gpg-agent with Emacs flatpak

13 April 2020

TL;DR: Enable socket-activated gpg-agent && add an override to the Emacs flatpak for --filesystem=xdg-run/gnupg:ro.

When I first installed Fedora Silverblue a couple of releases ago, one of the first flatpaks I installed was (obviously) Emacs. Unfortunately I switched back to the layered package pretty quickly, because any time I tried to do anything that involved GPG (including signing Git commits, tags, etc), I ran into trouble. I didn’t have the time to dive into the depths of flatpak at the time, but because I’m on lockdown for a long weekend, I do now!

The first thing that I learned when searching around the web (hoping that someone had posted something that I could just paste into a terminal), was that making use of gpg-agent from within a flatpak is not a solved problem. There are a couple of open issues against some specific flatpaks, and there are more general issues like this proposal to add a portal, and this issue to add a socket option.

I initially went down the path of trying to copy what some other flatpaks do: I built a custom Emacs flatpak with pinentry installed, & changed the default command to be a script that would start a local gpg-agent inside the flatpak sandbox instance, then run emacs, and kill that local gpg-agent before exiting. I even opened a pull-request to propose this be added to the main emacs flatpak!

Thankfully, after banging my head on several flatpak and gpg-shaped walls, I found what I think is a simpler way! Firstly, the magic override to give the flatpak sandbox enough permission seems to be the following (thanks to the Thunderbird flatpak README for the tip):

flatpak override --user --filesystem=xdg-run/gnupg:ro org.gnu.emacs

Next, while testing with that I found that if there isn’t a running gpg-agent, gpg inside the flatpak sandbox will have a bad time, and give an error like the following:

[gryan@localhost ~]$ flatpak run --user --command=sh org.gnu.emacs
[📦 org.gnu.emacs ~]$ gpg --decrypt .authinfo.gpg
gpg: can't connect to the agent: IPC connect call failed
gpg: encrypted with 4096-bit RSA key, ID 123D53E57C1D8F89, created 2019-04-05
      "Gerard Ryan <redacted@example.com>"
gpg: decryption failed: No secret key

Enabling and starting socket-activated gpg-agent for the user via systemd magically made it happy again. The systemd units for this aren’t installed into a usable location by default, but instead come shipped as examples in the gnupg2 RPM; so we need to make them available to systemd & enable & start the socket:

sudo ln -s /usr/share/doc/gnupg2/examples/systemd-user/gpg-agent.s* /etc/systemd/user/
systemctl --user enable gpg-agent.socket
systemctl --user start gpg-agent.socket

That’s it! Everything should work as expected now.

Bonus Emacs content!

With the above, if gpg-agent needs your passphrase from Emacs, you’ll get the usual GNOME 3 system prompt. If you’d prefer to be prompted in a more Emacs-native way, you can do the following:

Add allow-emacs-pinentry to ~/.gnupg/gpg-agent.conf
Install the pinentry package in Emacs
Add (pinentry-start) to your Emacs configuration

Testing Jenkins Pipeline shared libraries

25 March 2018

In this post I’ll talk about how to configure shared libraries so that they can be easily tested using with their own Jenkinsfile, and the traditional GitHub pull-request process. I won’t be talking about using JenkinsPipelineUnit since I’m not hugely familiar with that yet. Maybe I’ll write another post about that now that it’s included in a new maven archetype for shared libraries!

First a little bit about Jenkins Pipeline shared libraries. These are collections of reusable classes, and DSL scripts which act similar to steps provided by plugins. Unlike steps from plugins, they won’t show up in the 'Pipeline Syntax' page which is a really helpful tool for generating snippets when creating pipelines (although if you go to 'Global Variables Reference' from there, you’ll see them (assuming Jenkins has loaded your library at least once as part of a pipeline). They do offer advantages over plugin steps though: they’re easier to make changes to, and as this post will highlight, you can use different versions easily.

More info: https://jenkins.io/doc/book/pipeline/shared-libraries

Putting shared logic for pipelines into a shared library allows us to keep our pipeline Jenkinsfile contents clean and easy to read, and also we only have to update in one place when we want to change that logic for all jobs. One problem is that the phrase making changes that affect many pipelines can also be written as making changes that break many pipelines. Why can’t we use the same development flow to make changes to our Pipeline shared library as we use in all of our other projects? It turns out we can!

The library definition in the global Jenkins configuration

In the Jenkinsfile for the shared library, we’ll want to be able to use the 'library' step, rather than the '@Library' annotation, and we’ll want to be able to specify a Git branch or GitHub pull request. This will require making some changes to the definition in the global configuration. Specifically, we’ll want the following:

Uncheck 'Load Implicitly' (beware that this could affect existing jobs)
Check 'Allow default version to be overridden'
In the GitHub configuration, add behaviours like in the image below ('Discover Branches', 'Discover pull requests from origin', 'Discover pull requests from forks'):

Figure 1. Configuring a GitHub-hosted shared library in Jenkins

The first two choices here ('Load Implicitly' and 'Allow default version to be overridden') will allow us to use the library step to choose a different version of this library than the default. In this case, we’ll have a CI job for the shared library defined in a Jenkinsfile in the repo, which will get triggered on a GitHub pull request. The version of the shared library we want to use in the CI job, is the version from that same pull request — this is what the additional behaviours in the GitHub section allow us to specify!

The Jenkinsfile in the library repository

node {
    // Load your library!
    def pipelineLibrary = library("my-pipeline-library@${env.BRANCH_NAME}")

    // We store a reference to it in a variable, so that we can
    // instantiate classes from it like this!
    def utils = pipelineLibrary.org.feedhenry.Utils.new()

    stage('Test a global variable) {

        // A global variable doesn't need to use the library variable,
        // since it's global
        String myResult = sayHello('Joe')
        assert myResult == 'Hello Joe!'
    }
}

To explain the above a little:

Since we’re using the 'GitHub Branch Source' plugin to create jobs for the branches and pull requests for the projects in our organization, this Pipeline could be run from a pull request, or from a regular branch. Wherever it’s from, we want to use the library from that same ref, and that’s what we determine and store in the String 'gitref'.
Holding a variable reference to it will allow us to instantiate classes from the library, such as the Utils class in the above example.
We can use 'global variables' such as 'sayHello' above.
Using the Groovy 'assert' keyword, we can test classes and global variables directly in the Pipeline!
If your shared library is in a public repository, you might want to protect this job from people who just want to use it to get your Jenkins instance to do their bidding. See my previous post about adding a 'trust' stage to your Jenkinsfile.

Pre-merge testing against other jobs

By default, the library version that you’ve specified in the global configuration will be what’s loaded in your pipeline. If that’s a branch (convenient), then once you merge your changes back to the branch, those changes will apply to future pipeline executions. Wouldn’t it be great to be able to try it out in affected pipeline jobs one by one, rather than breaking them all at once?

If it’s possible to test those other pipelines without having to worry about their side effects, or some rigid process around them, then you can — no need to merge your change first and risk breaking many pipelines!

Rembember the GitHub repository configuration we defined for the library in the shared configuration? You can make use of that in the other Pipeline jobs by specifying the pull request (or other branch) when loading your library there too. When we’re manually testing them out like this, we know the ref we want to use, so we can use that directly in the @Library annotation, if that’s how the Pipeline’s load it.

To test out a custom version of the shared library in another Pipeline that similarly gets loaded from a Git repository using the "GitHub Branch Source" plugin or something similar, you could create a dummy pull request to that repository, which just changes the version of the library to that of your library pull request. Otherwise, you could use the 'Replay' functionality in the Pipeline job in Jenkins to modify the Pipeline for your once-off execution.

Trusting Jenkins Pipelines in public repositories

11 March 2018

We use Jenkins for a lot of things in the projects that I work on. One of those things is CI jobs for open source projects that we maintain on GitHub, which run automatically on pull requests for these repositories. Previously we used Jenkins Job Builder to manage the jobs, and the GitHub Pull Request Builder plugin to trigger them on different events in the pull request. Now however, we prefer the expressiveness of Jenkins Pipeline, and the ability to define the job in a Jenkinsfile in the target repository, managed in Jenkins by the GitHub Branch Source plugin. In this post I’ll describe how we get a similar trust model (run builds for 'trusted' users, but not for others without approval from a trusted user) in our new configuration.

One great thing about the GitHub Pull Request Builder plugin is that it determined whether or not to automatically trigger a job, based on the trust of the user who created the pull request, using white and/or black lists. If you’re maintaining your own Jenkins instance, there’s a good chance you don’t just want it to execute code on behalf of untrusted people! The event of a not-yet-trusted user creating a pull request would result in a comment on the PR from Jenkins, asking for someone trusted to verify that it was okay to run the build; as opposed to the build getting run immediately for someone who was considered trusted.

The GitHub Branch Source plugin does things differently: it will automatically trigger builds for all pull requests or none (actually, you can choose to build pull requests from the origin repo, and not from forks; but we generally prefer to have all pull requests from forks).

If a pull request comes from an untrusted user, it will automatically trigger the same as from a trusted user (your choice of trusted can be 'Everybody', 'Nobody', 'Contributors', or 'users with Admin or Write permission'), but it will protect from malicious changes to the Jenkinsfile by using the version of that file from the trusted branch (e.g. target branch of the pull request).

In the untrusted case (we choose to trust 'Contributors'), you’ll see a line such as the following at the top of the console output of the run:

Loading trusted files from base branch master at 72204ebf115acfd9a3582b708939c1353d3eae75 rather than 7382ebd0b32cdceb95c7afb27d8a325f17953536

For some use cases this is enough, but if you execute other scripts or tools from the build, couldn’t Mallory just get their wicked way through those? Yes, you could do a readTrusted on each potential file that could be used to execute arbitrary code, but I imagine that would get inconvenient quickly, and it might be easy to miss one.

For these cases, we’ve got an initial 'Trust' stage, which runs before anything else in our public pipelines. It just checks to see if the person who submitted the changes is a member of the GitHub org. If they are, it will continue; but if not, it will wait for someone who is trusted to log in and hit the 'Proceed' button on the input step. As mentioned, because the Jenkinsfile will be loaded from the trusted branch if the author is untrusted; malicious authors can’t just remove or modify this stage!

To keep our Jenkinsfiles clean, we’ve got the logic for this in a shared library, so this 'Trust' stage just contains one line:

Example Jenkinsfile

#!groovy

// https://github.com/feedhenry/fh-pipeline-library
@Library('fh-pipeline-library') _

stage('Trust') {
    enforceTrustedApproval('aerogear')
}

stage('Actual stuff') {
    node('probably') {}
}

The actual logic in the shared library can be found here.

Two things to note:

Since it will be executed against regular branches on the origin repository, just as it is for pull request projects; it automatically trusts those branches: it will only query for GitHub org membership if the CHANGE_AUTHOR environment variable is set (it is for pull request projects, not for normal branch projects).
If your shared library project is also a public repository with a Jenkinsfile that similarly runs on pull requests, and if you load the library version from the pull request commit; either do a readTrusted on this file, or duplicate the function into the Jenkinsfile.

Our shared library is available under the Apache-2.0 license, so feel free to copy the relevant file (or any others) to your own shared library (this is probably a better idea than depending on ours). Additionally, if you find any issues or have any improvements, we would love to hear about them!

My Podcast list in 2016 🎧

09 July 2016

I’ve started listening to podcasts again, as I’ve had a commute of around 40 minutes in the last few months. Unsurprisingly, the majority of the podcasts that I listen to frequently are focused on free software or software development in general. So here’s a list of some of the shows that I listen to regularly now. For my full podcast list (including ones not listed here that I haven’t really gotten around to listening to yet), feel free to download my OPML file here. I’d love to hear about what’s missing from the list, so feel free to suggest a new one to me!

TL;DR Navigation

Software development
Free software / Free culture
General Interest

Software development

The Changelog

Homepage | RSS | iTunes

The Changelog is a show about usually open source software projects and communities that are interesting to developers / engineers. The format is usually an interview with a community or project leader, and they get some great guests. The high quality interviews make it a popular show, which in turn attracts these high profile guests, so the quality and popularity keep rising! I listen to this show a lot, but I sometimes skip episodes that are on some specific topic that I’m not currently interested in. Instead, I often end up scrolling back to older episodes from before I started following (there are over 200 now!).

Ruby Rogues

Homepage | RSS | iTunes

Despite the name, and the description on their site, my experience with this podcast is that it’s not all that ruby-specific — which is great, because I really like it, but I’m not a Ruby developer! It’s more of a panel discussion show with a cast of hosts and an expert guest on a particular topic; where many of the topics are cross-cutting aspects of software engineering, rather than a specific project or ecosystem, making it applicable to all types of engineers involved in software production. This is definitely a favourite of mine at the moment, and when I run out of things to listen to, I go back and find older episodes in the archive that I haven’t heard yet.

Software Engineering Radio

Homepage | RSS | iTunes

SE-Radio is similar in ways to The Changelog. I haven’t been listening to it for very long, but in that time it has risen high in my list of favourites. It seems to be usually a 1:1 interview with a guest from a given project or community or company or research field or somewhere. To give you an idea, some of the topics recent topics have been Ruby on Rails, Alluxio, OpenStack, recruiting, and Clojure.

NodeUp

Homepage | RSS | iTunes

At my current job at Red Hat, we use NodeJS heavily. The NodeUp podcast is a great show for anyone interested in hearing about what’s going on in the NodeJS community! It tends to be a conversation between a host or two (rotated) and two or three guests, around some aspect of the NodeJS ecosystem that the guests are involved with.

FLOSS Weekly

Homepage | RSS | iTunes

This was one of the first to get added to my current list, as I used to listen to it before. It’s a weekly show with the format being two hosts (usually Randal Schwartz and another rotating host) interviewing the community lead(s) of a given free software project/product. It’s also streamed live (and recorded I believe) as a video cast, but I have to admit to never having viewed it. As a show that has been going for over 10 years now (!), they’ve got a back-catalog of almost 400 shows now, so there are many gems to be found in the archive.

The InfoQ Podcast

Homepage | RSS

This is a relatively new podcast, with only 7 episodes at the time of writing. It comes from the people behind the InfoQ site and QCon conferences. If you are familiar with the InfoQ site, you might have an idea of the breadth of topics that it covers — not all are interesting to everyone, but at least some will be relevant to some engineers. The podcast isn’t as busy as the website, but I imagine that the episode topics will vary to the same degree. Definitely worth following so that you don’t miss the episodes that interest you, even if not all do.

Thoughtworks Beacon Podcast

Homepage RSS

You may have heard of Thoughtworks, the software consulting company who publish the Thoughtworks Technology Radar every few months, with their thoughts on trends in the industry. The podcast is an interesting one, as it doesn’t try to stick to a defined format or length. Many of the episodes are shorter thoughts on a particular topic, and others are longer interviews or discussions. Some episodes sounds like they may be scripted, which (if so) is interestingly different, and makes these episodes more like recorded blog posts. I don’t find that to be an issue at all though: it doesn’t detract from the quality; and after all, there aren’t any hard format rules that podcasts have to stick by, are there? :)

Free software / Free culture

Linux Voice

Homepage | RSS (mp3) | RSS (Ogg)

Yes, it’s those people that make the magazine! I used to listen to the TuxRadar podcast, which is the predecessor to this (it’s just rebranded after the move to the new magazine), so this was an early addition to the new list. It’s structured into different sections, which is a nice break from the common interview format of many of the podcasts that I listen to:

News - what has happened in the world of GNU/Linux and free software since the previous episode
Finds of the fortnight - each of the hosts talks about something interesting that they’ve learned about recently.
Vocalise your neurons - Listeners can send a rant or rave about something and email it to Mike to read, and the hosts discuss.
Voice of the masses - A question is asked for the community to give their opinions on through the comments on the website. A sampling of the opinions is provided on the show and discussed.

Bad Voltage

Homepage | RSS (mp3) | RSS (Ogg) | iTunes

Similar to Linux Voice, in that it’s a constant panel of hosts talking about what’s new in this crazy world since the last episode. The hosts are pretty high profile individually, so there’s always lively debate. Sometimes there are guest cameos for particular segments, and often a gadget review.

Hacker Public Radio

Homepage | Feeds (many options!)

HPR is a feed that I follow and listen selectively rather than trying to consume every episode, as there’s a lot (notice how they’ve got split feeds for 'latest' and 'all', with warnings attached to the 'all' feeds due to the size)! Episodes are submitted by community members, and can be any length. The only guideline is that topics should be of interest to hackers.

Free as in Freedom

Homepage | RSS (mp3) | RSS (Ogg)

I follow this show, but it currently appears dormant. It’s usually a discussion amont the esteemed hosts, Karen Sandler and Bradley Kuhn, about free software licensing and other concerns; with the occasional guest.

General Interest

Freakonomics Radio

Homepage | RSS | iTunes

This is a very professionally prepared podcast compared to most that I listen to — it’s clear that a lot of work from more people than just the host goes into each episode. I’m sure the fact that it airs on public radio in the US helps to fund that. The topics here are taken from a very broad spectrum, such as Leicester City football club winning the English Premier League, or religion, or suicide, or sleep, or anything else.

Star Talk

Homepage | RSS | iTunes

This is another one that’s also a radio show (and now a TV show?). I’m not going to bother describing this one, since it’s probably one o f the most popular podcasts around.

Ted Talks (audio)

Homepage | RSS | iTunes

Yes, there’s a curated list of Ted talks that’s edited for audio-only consumption, and this is it! The episodes are usually shorter than a lot of other podcasts, as it’s just one person giving a live talk. The shorter length is nice in my experience; as although topics aren’t explored in a huge amount of depth, they’re more digestable than a longer show (maybe it’s just that they fit more easily into my commute).

Two Pump.io client apps for Fedora (COPRs)

26 January 2015

Pump.io is a "social server with an ActivityStreams API". It’s from the same people that originally created StatusNet, which has now become GNU social. Pump.io servers are nodes in a federated social network: when you have an account on one, you can interact with others on that server, but also others with accounts on other pump.io servers. It’s free software, so you can host your own instance and connect to others, wherever their servers may be.

See the Pump.io User Guide for more information and links.

The default method of using Pump.io is through a web interface that comes as part of the core software. This is good, but there are a few rough edges here and there. There are a few android applications, such as Impeller, Puma, and AndStatus.

More importantly for this post, there are a couple of nice desktop applications, two of which are packaged in COPRs for Fedora, made by HowCanUHaveMyUsername!

Dianara

Dianara is a Qt-based application and is quite customizable. To add the COPR, run:

sudo dnf copr enable howcanuhavemyusername/Dianara

then to install:

sudo dnf install dianara

Pumpa

Pumpa is also Qt-based, but has a more minimal default interface than Dianara. More information on Pumpa is available here. To add the COPR, run:

sudo dnf copr enable howcanuhavemyusername/Pumpa

then to install:

sudo dnf install pumpa

If you want to connect to some interesting people on pump.io here are some good starting points directly related to this post:

Jan Kusanagi made Dianara
Sazius made Pumpa
Evan Prodromou made Pump.io
Howcanuhavemyusername (Metal Biker) made the COPRs
George Standish is in the screenshots
lostson is also in the screenshots
Gerard Ryan (that’s me!) made this blog post

Prank - alias sudo in BASH (or why you should start escaping privilege-escalating commands)

11 January 2015

A little while ago, a friend and I had an idea for a prank to play on someone who leaves their Linux-based workstation unlocked and unattended (I guess this would work for OS X also, but I don’t have enough experience with that to say for sure).

The goal was to write a line to the target’s /etc/hosts file. For this, we would need privileged access. Now, we know the target well enough to know that they use the familiar sudo command for various tasks, such as installing new packages. We pondered a while on how we could make it work, and came up with a possibility: what if we could set an alias for sudo in the users BASH configuration file, that would executed our desired command?

We figured out this part quite quickly, but then pondered on how we could improve it: what if we could delete the evidence after executing? The perfect crime.

The following was our final incantation in ~/.bashrc:

alias sudo='/usr/bin/sudo sh -c "echo \"127.0.0.1 twitter.com\" >> /etc/hosts" && unalias sudo && sed -i -e "s/alias sudo.*//" ~/.bashrc && sudo'

One way to defend against being fooled by this humourous prank (or malicious attack?) is to prepend a backslash before sensitive commands such as this (so sudo becomes \sudo). This tells BASH to ignore aliases, and use the command directly from the PATH. The question comes to me though, should certain commands be restricted from aliasing?

Our intentions were innocent and light-hearted (and our results were glorious!), but one can see how malice could easily be introduced.

Fedora Java Spin - GSOC 2012 update 13 (Final)

20 August 2012

This is the thirteenth weekly instalment of my GSOC blog series. More info on the progress of the Fedora Java Spin/Remix can be found on the Fedora wiki.

Today is the GSOC Firm ‘pencils down’ date, as opposed to last weeks ‘Soft’ pencils down date. More info on the hardness of pencils can be found here.

This week, being the last, there are new Live images available. I got a lot of help from Dan Allen this week, and we focused mainly on making the live distribution as great as it can be. We’ve decided to name these as a Remix rather than a Spin, since it’s still not an official Fedora Spin. I think it’s pretty great, but there’s always room for improvement, so if you think something would be better done differently, please let us know!

The ISO image files can be found at ~~http://ryan.lt/fedora-java-remix/~~ (lost due to disk failure - 2013/03/25)

Here’s what you can expect:

Standard Fedora GNOME desktop experience, with some minor customizations aimed at developers, including:
- GPaste extension for GNOME Shell.
- Caps Lock key is an additional control key.
- Custom Firefox homepage with relevant search feature, and additional useful bookmarks for Java developers.
- Custom .gitconfig with some useful shortcuts.
Additional Java and related packages from the Fedora repositories, including:
- OpenJDK
- JBoss AS7
- Groovy
- Scala
- Maven
- Apache Ivy
- Thermostat
- Tomcat
- and more.
Useful tools for developers (Java or otherwise):
- Git & Gitg,
- SVN & RapidSVN,
- Mercurial & TortoiseHg,
- Meld,
- vim,
- emacs,
- source-highlight,
- and more.
Eclipse IDE with some useful plugins and customizations, including:
- Eclipse Webtools (those that are available in Fedora repositories).
- JBoss Tools (those that are available in Fedora repositories).
- EclEmma
- Eclipse plugin for FindBugs
- Eclipse Mercurial plugin
- Eclipse Git Integration
- Eclipse Subversion plugin
- JBoss AS user instance pre-configured to be managed by JBoss Tools in Eclipse.

So that’s it. GSOC is over. I’d like to give special thanks to my mentor Marek Goldmann; and also to Dan Allen who were both extremely helpful and supportive the whole way through. Huge thanks also to all of the people from the Fedora Java SIG who were always able to advise me as to what the best course of action would be in all of the stumbling blocks that we ran into; and to the JBoss Tools team for their helpfulness and expertise where and when it was needed. I look forward to continue working with everyone into the future, as we make Fedora the first choice for Java developers.

My current goals coming out of GSOC are to continue what I’ve started, including getting more JBoss Tools built (which again, still requires m2e-core, which I still haven’t figured out why it doesn’t work…grumble).

Once the Java Remix gets accepted as an official Fedora Spin, I hope to get more involved with the Spins SIG. I think spins tailored to specific audiences often makes more sense for people than one catch-all distribution. I’m surprised that there aren’t far more Spins, since the tools and documentation available are pretty great, and make the process a whole lot easier than it could be. I’d like to find out what the best way to include custom browser profiles in a Spin would be. A browser is an essential tool for any modern user, and I feel that since Spins are often targeted to a specific audience, their browser experience could be better customized also.

That’s all for now. Thanks again to everyone who has been involved or supported/ motivated me along the way! I hope I pass the final evaluations…the tee shirts that are given to successful students are usually great and I really want one (oh, the financial stipend would also be nice)! :)

Fedora Java Spin - GSOC 2012 update 12

13 August 2012

This is the twelfth weekly instalment of my GSOC blog series. More info on the progress of the Fedora Java Spin can be found on the Fedora wiki.

Holey moley, it’s the 13th of August! That means that it’s the ‘Suggested pencils down’ date for GSOC 2012, with August 20th being the ‘Firm pencils down’.The complete timeline can be seen here. On that note, I expect this is the penultimate entry in this blog series, after that I’ll go back to posting stuff of little importance! I’ll admit, writing the blog posts haven’t been my favourite part of the process, but they have been quite beneficial to keep me focused.

A nice surprise this week, with someone else packaging one of the dependencies on the list: Alexander Kurtakov (Eclipse Guy!) prepared eclipse-swtbot which is needed for JBoss Tools tests. Thanks Alexander! Unfortunately tests still aren’t working, but I think it’s a problem with tycho version in Rawhide rather than failing tests. The eclipse-swtbot isn’t in Fedora 17 yet, so I’m unable to test with a stable tycho release yet.

Next on the list was eclipse-m2e-core, which I’ve mentioned a few times previously. I spent quite a while talking about it in last weeks post so I’ll keep it brief this week. I thought I would have this ready for review very quickly this week, but it somehow dragged into Wednesday or Thursday. A review request is in, but there are issues. I don’t know if the issues exist in the spec that will be reviewed for Rawhide as I can’t currently test it. I got it to build in F17 with about twice as many patches; but after all that and getting it to build, guess what, it didn’t work! Eclipse just decided to ignore most of it for some reason. I’m hoping that it’s some small issue that I’m just overlooking, or maybe something in those extra patches that’s causing a problem. I spent hours looking at it to no avail. Akurtakov said he would take a look at it on Monday, so I’m hopeful of that. No better man for the job!

The reason I was packaging m2e (apart from it being a totally useful and cool thing to have anyway), was for JBoss Openshift Tools. Even though I’m not sure if m2e works, I’m sure that it builds; so I decided to try to build eclipse-jbosstools-openshift, now that I could use it as a dependency. Success! After a patch or two, it built. I’m really excited about this, because I use Openshift (this blog is hosted on it, for example), so I’ll be able to use these tools! I’ve never actually used any of the other JBoss Tools stuff (shh, don’t tell anyone), but I’m looking forward to learning to use them once I have time!

I also cleaned up the specfile for eclipse-jbosstools quite a bit. Now it’s a leaner, meaner, package building machine spec file. It used to take more than minutes to do a build on my computer, now it takes a little over 5. I think that’s a pretty drastic improvement. I also cut the number of patches almost in half. Using the new POM macros makes it far more maintainable I think!

Oh crud, I meant to make a list of JBoss Tools modules that are in Fedora, ones that build but are waiting, and ones that don’t currently build. Oh well, tomorrow is another day!

Fedora Java Spin - GSOC 2012 update 11

06 August 2012

This is the eleventh weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

This week saw the inclusion of eclipse-jbosstools into the Fedora package collection. It’s currently available in the updates-testing repository if you would like to try it out and provide some feedback! Not all of the JBoss Tools are packaged yet, but if you use some of the available ones, I’d love to hear of any problems or successes that you have with the packages!

Next on my list was to submit the spin to the Spins SIG. As you can see from the title of this post, the name has changed from JBoss spin to Java spin: hopefully it will appeal to a broader spectrum of Java developers than just those who work with JBoss related stuff. Although I submitted it by the date that I originally thought was the deadline for getting a spin reviewed/accepted for F18, looking at info on the wiki today makes me less sure that my deadline was the correct one: the spins process page says that spins need to be submitted at least 3 weeks before feature freeze, rather than the 2 weeks that I originally thought. F18 release schedule also lists feature freeze as 2012-08-07, and not 2012-08-14 as I originally thought. When I realised this I contacted the spins SIG to see if it was too late to get into F18; but as that was only this evening, at the time of writing this, I don’t yet have confirmation either way. If it is too late, the name might be changing once more, from Java Spin, to Java Remix (unofficial spin).

After that, I went back to trying to build eclipse-m2e (Maven integration for Eclipse), a process I had started weeks ago. I ran into some strange problems that took a while here! I got some help from rgrunber in #fedora-java with an odd problem I was having where the jar for maven-model from maven2 was always being chosen over the more up-to-date maven-model from maven 3, and preventing build. Solution is to pass a depmap explicitly as an option to the maven invocation. An example of this can be seen in the spec file for tycho. Oddly enough the reason tycho uses an explicit depmap file is for the same problematic jar – maven-model. Perhaps something should be changed so that the up-to-date version is chosen by default?

The next problem had me stumped for about a day and a half: tycho was just giving up half way through, displaying an eclipse dialog that spanned my entire screen with a message that started with ‘JVM terminated.’ Absolutely no idea what was going on, had I. After trying close to everything, I eventually figured out what the problem was. Of course, solving problems is a lot easier when you know what the problem is! :) It turns out that it wasn’t something that I did (that’s always my first assumption), it was a problem with the jbosgi-repository package that had already been fixed in rawhide, but had not yet been pushed to F17. Fantastic.

With those hurdles out of the way, I eventually got to building m2e-core again. Today, I managed to get it to build successfully! I’m currently making a spec file for it, and hopefully I’ll have it ready for review very soon. I thought that it was going to involve patching OSGI information into every jar that it bundles, but after making a list of all of the jars that I expected to have to patch, I had a realization that most of them would not have to be touched at all. I’m actually only after making that realization right now, I hope I’m right!

Fedora JBoss(?) Spin - GSOC 2012 update 10

30 July 2012

This is the tenth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

I mentioned last week that I had a few packages that needed reviewing before the JBoss Tools package could be reviewed: all of those got reviewed by some of the experts from the Fedora Java SIG. For most, there were a couple of things that needed to be changed, but this isn’t a bad thing, since it’s the part of the process that I learn the most from!

After I got the latest release of JBoss Tools downloaded, I tested building from that. Luckily, there was very little that needed changing! Once all of the newly reviewed packages made their way into rawhide, I tried doing a koji scratch build, which failed miserably. I eventually found what the source of that problem was: it seems that the eclipse-jbosstools package will be architecture dependent (i.e. there will be a slightly different set of rpms i386 and x86_64. The error message that was being displayed was something like ‘Unable to determine SWT fragment’ or something like that. I had completely forgotten that I had put something in the parent POM file many weeks ago to get stuff to build with rpmbuild. Currently there are slightly different things being done, depending on the architecture.

By the time that I was ready to submit the updated/working srpm for review, I had moved to my parents house to take care of the house and dog (Sorry internet: dogs > cats) while they are away for a few days. Apart from the inconvenience of getting here and setting myself up again, there’s only really one big disadvantage of being here: a painfully slow net connection! For this reason, I had to cut down the size of the tarball inside the srpm to allow me to upload it. Reducing it from \~460MB to ~77MB made it a little easier – only taking around 45mins to upload, rather than several hours!

I hope to get this package reviewed tomorrow (Monday) so that I can finally submit the spin for review. I did a little more work on the kickstart file for the spin this week also, and I’m very close to getting the Application Server to be managed by JBoss Tools by default in the live spin. More work on that tomorrow, then submit it for review!

The name of the spin has also been brought up in discussion this week. It is to be decided whether it will be submitted as Fedora JBoss Spin or Fedora Java Spin. More on this in the next post I’m sure!

Fedora JBoss Spin - GSOC 2012 update 9

23 July 2012

This is the ninth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

This week’s positive points:

We have what I would like to call a temporary fix for the jboss-reflect issue that I talked about last week. The problem was that the latest version forked another library and bundled it’s own subset of that. Our temporary fix is that we’ve now got an older version that doesn’t have any offensive stuff in it, and works fine for what we need it for. Thanks to Marek Goldmann and Mikolaj Izdebski for their help with this! Huge thanks also to Rob Stryker who is working upstream to remove the dependency on this! :)

eclipse-wtp-jsf is packaged and awaiting review. This is great because it allows two more parts of JBoss Tools to be built: WS Tools and CDI Tools. I think eclipse-wtp-jsf may also be my first package that didn’t need any patching to be built for Fedora! This probably isn’t newsworthy, but I found it quite odd, considering some of the heavy patchwork that was needed for some of the other wtp packages.

Along with WS and CDI from JBoss Tools, the Freemarker IDE module is also building. This probably could have been included at an earlier stage, but it just clicked with me this week why it wasn’t able to import the necessary stuff that it needs from the freemarker jar that’s included in Fedora already.

I think the deltacloud module will also work, now that I have the deltacloud-client-java package created and waiting for review. I’ve only just created that package though, and I’m currently downloading the new 3.3.1.Final release of JBoss Tools so I don’t want to do any more local builds tonight until I can use that (since building JBoss Tools takes a while!). Also, have I complained before about the poor connection I get to anonsvn.jboss.org? It takes me hours to download a tag of JBoss Tools. :-/

This week’s not-so-positive points:

I was hoping to get the spin submitted for wrangling with the spins sig by now, but that hasn’t happened yet. There are a couple of packages that still need to be reviewed before I can even submit the amount of JBoss Tools that I included in the iso that I released for the last milestone two weeks ago (in other words, not including the modules that I’ve mentioned above). If you’re reading this, then you’ve got far too much time on your hands already: go review a package from the list! :)

I got a little bit closer to getting m2e-core to build, but I’m running into some problems that I don’t understand. I think they are related to tycho, but it’s probably something that I’m screwing up somewhere. I think I probably spent too long on this, and eventually gave up on it. I think focus needs to be on getting the spin kickstart file in for review now, as it needs to be submitted at least two weeks before F18 Alpha freeze if we want any chance of having it as an official spin for F18.

Hmm, that was a little shorter and to the point, compared to my last few posts on the subject!

Fedora JBoss Spin - GSOC 2012 update 8

16 July 2012

This is the eighth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki. The fact that there is a blog post this week is good news, because it means that I passed the midterm evaluation, so I get to keep doing this for the second half of the GSOC period! :D

Progress

I started off this week looking into packaging m2e-core, which is a necessary requirement for some parts of JBoss Tools, including the examples from jboss.org/jdf and openshift integration; both of which would be great additions to the spin I think. After spending a while working with this, it became apparent that lots of patches would be needed for existing packages, mainly to add OSGI information. I started making some of these patches, but I decided not to start submitting them until I got a working spec file for m2e-core. This did not happen this week, because other accidental stuff (below) became a higher priority. I did manage to get two dependencies for it packaged, reviewed, and included in rawhide and f17: port-allocator-maven-plugin and maven-wagon-ahc; and a third packaged, which is awaiting review: maven-indexer.

Two steps forward, one step back

Going back to what I had done last week, Mikolaj Izdebski began to review some of the stuff that I had waiting for review. A significant issue was found with jboss-reflect (which is a dependency of jbossxb, which is a dependency of org.jboss.ide.eclipse.archives.core): it contains a forked objectweb-asm, something that I had completely overlooked. As this is not allowed in Fedora, it would have to be removed. I first tried just deleting the relevant files, but needless to say that didn’t work! I then spent a while looking to see if the stuff that isn’t in the upstream objectweb-asm could be added directly to where it’s needed in jboss-reflect, but I didn’t get very far with that, and I believe it would be a horrible solution anyway! Looking at jboss-reflect, it seems that it may no longer be actively maintained, as the only place I could find source code for it was as a subproject of JBoss AS6. If this is the case, I think the optimal solution would be to not package jboss-reflect, if at all possible. To this end, I have sent an email to the jbosstools-dev mailing list to see if there is any possible workaround for using jbossxb (as it would also be good not to have to package that!), or to try to elicit other possible solutions that those guys might be able to think of!

A little rant about github

I also had my first real experience using github this week, when I sent pull requests to sonatype to include license files for both port-allocator-maven-plugin and maven-wagon-ahc. I don’t fully understand why it is so popular amongst so many free and open source developers, when it is a non-free platform compared to alternatives such as gitorious, which is 100% free software under the AGPL. Seriously, what is the must-have killer feature of github? I mean I get that it’s shiny and social and stuff, but it seems odd and a little disturbing that such a large part of the greater open source community is not defaulting to the more open alternatives. That’s the main reason that I decided to put the repository for the Fedora JBoss Spin kickstart onto gitorious instead.

eval(GSOC)

As I mentioned at the very beginning of this post, I passed the midterm evaluation for Google Summer of Code 2012. The evaluation period ran for most of the past week, finishing on Friday, when all of the evaluations had to be submitted. As part of the evaluation process, I had to submit an evaluation of my experiences over the first half of the program. Marek also had to submit an evaluation on his experience as my mentor: he must have said at least some good things since we passed…thanks Marek! :D

Eclipse Webtools updates

I eventually got around to updating the Eclipse WTP packages that I maintain, to their stable 4.2.0 versions. Of course, not without running into a few little snags! This was the first time that I had updated a package to a new version, so I got to learn how that works. It took me a while to figure out why the newly created sources tarball that I had created wouldn’t upload with the fedpkg new-sources <tarball> command, until I realised that the user I was issuing the command with, didn’t own the file that I was trying to upload. I’m not sure why that doesn’t work, but it’s probably for very good reasons! I also had to create proper scripts to download the relevant stuff from the eclipse webtools map file. I may have mentioned before that I find it hard to follow the organizational structure of the webtools project. When I remember that they produce a map file for each stable or development milestone release, then I don’t feel so bad. Most of the plugins and features here are hosted in CVS repositories, but there are some on SVN and some on git also I think. OF COURSE this caused me problems! :) There is one bundle foe org.eclipse.jpa that needs to be grabbed from SVN, whereas all the rest come from CVS, so this one had to be handled differently. Originally I just checked out the development trunk, which made everything fail spectacularly. I didn’t realise why immediately, but I eventually realised that I needed to get the revision information from the map file manually for that one.

Thats all for this week folks, tune in next week for more Fedora JBoss Spin fun and frolicks…same bat time, same bat channel!

Fedora JBoss Spin - GSOC 2012 update 7

09 July 2012

This is the seventh weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

Today is the ‘mid-term’ in the GSOC 2012 calendar, and as I mentioned last week, I’ve got the next iteration of the the spin ready. There are a few caveats to it, but I think it’s pretty cool so far! :) Not all of the packages are in the Fedora repositories yet, so I created a local yum repo that I could include the missing packages through. +Dan Allen added some ideas to the wiki for customizations that will make the spin awesome. Although these are only ‘proposed’ so far, I’ve included some of them into this release to see how well they work. A lot of them are not possible yet, and some of them I just can’t figure out yet; but the remaining ones should be in there! Here’s a vague list of stuff that the spin has over the vanilla fedora desktop image:

Note

JBoss AS7
Eclipse, with all of the available webtools from Fedora, and lots of other features.
A subset of JBoss Tools (this isn’t in Fedora yet)
Some SCM tools, such as gitg, TortoisHg, and RapidSVN.
Various other Eclipse customizations, including keybindings and JBoss perspective by default
Gnome-Terminal opens maximized by default, and has some custom keybindings (see Dan’s recommendations)
JAVA_HOME is set in ~/.bash_profile (I think, I forgot to check if this worked!)
Caps Lock key functions as an additional Ctrl. No more cruise control.
The proposed Gedit customizations, apart from the custom keybind
Attached modal dialogs are disabled by default.

The current set of JBoss Tools includes the following components:common, usage, archives, jmx, as, and jst. There are a few more that are close to being ready to include, and hopefully it won’t be too long until all of them are included! My main disappointment is that we don’t yet have m2eclipse to provide maven integration in Fedora, so the quickstart examples from jboss.org/jdf can’t be added in the recommended way. I’ll look to package m2e soon, so hopefully this situation will change soon!

Here are some steps provided by Dan Allen, to connect to the Fedora packaged jboss-as, which works with the included JBoss Tools:

User instance, managed by JBoss Tools

Create a new user instance: jboss-as-cp -l $HOME/applications/jboss-as-user-instance
New > Server
Server name: JBoss AS 7.1 Runtime (managed instance)
Server runtime environment: Add… (if necessary)
Name: JBoss AS 7.1 Runtime (system), Home directory: /usr/share/jboss-as, Configuration file: standalone-web.xml
Click Finish (three times)
Double click on “JBoss AS 7.1 Server (managed instance)” in Server tab
Click the Open launch configuration link
[ ] Always update arguments related to the runtime.

Modify VM arguments:

Remove:

“-Dorg.jboss.boot.log.file=/usr/share/jboss-as/standalone/log/boot.log”
“-Dlogging.configuration=file:/usr/share/jboss-as/standalone/configuration/logging.properties”
“-Djboss.home.dir=/usr/share/jboss-as”

Append:

“-Dorg.jboss.boot.log.file=${env_var:HOME}/applications/jboss-as-user-instance/log/boot.log”
“-Dlogging.configuration=file:${env_var:HOME}/applications/jboss-as-user-instance/configuration/logging.properties”
“-Djboss.home.dir=/usr/share/jboss-as”
“-Djboss.server.base.dir=${env_var:HOME}/applications/jboss-as-user-instance”

Click OK
Select the Deployment tab
(*) Use workspace metadata
Save
Drag application to “JBoss AS 7.1 Server (managed instance)” in Server tab
Click the play button in the toolbar of the Server tab

Finally, the image files for this version of the custom spin are ready. That took quite a while to upload!

i686: outdated link removed

x86_64: outdated link removed

kickstart file: outdated link removed

Fedora JBoss Spin - GSOC 2012 update 6

02 July 2012

This is the sixth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

The first thing that I did this week was to find out if the extra features that I had proposed to be added to the eclipse-wtp-sourceediting package would be okay, given that they created a cyclic dependency. As I had expected, it was a big no-no! We discussed for a while in #fedora-java on freenode, what the best way to package features from the Eclipse Webtools project for Fedora, given that the structure of the Webtools CVS repositories doesn’t make much sense sometimes. Currently the wtp packages in fedora are separated into what one would think are subprojects: the next layer of sub-directories. Some alternative ways that were suggested were to have one package per feature, (or per core/UI feature pair, where those exist); or have one super webtools package, with everything subpackaged. None (including the current method) seem perfect.

I separated the problematic feature out into its own package. The feature was org.eclipse.jst.web_core.feature, and I already knew that the org.eclipse.jst.web_ui.feature would be needed further down the line, so I included that in the package also, and called it eclipse-wtp-jst-web.

The next thing that needed to be addressed this week was the bundling of unpackaged jars in the JBoss Tools ‘archives’ module. I got to work building those, starting with jboss-common-logging-spi. I got that built, packaged, requested review; and then I learned that it was not to be packaged, because it was obseleted by jboss-logging. Great. I spent a while trying to figure out where it was needed, and trying to see if the jboss-logging package would work as a drop-in replacement. I eventually figured out that it wasn’t needed by that particular plugin at all, but rather another one of the bundled jars. There were a few like thise, and one that depended on those was jboss-xml-binding.jar. The bundled version of this was old, so I had a look at the most up-to-date version, and found that I could use that with jboss-logging. Problem solved. Well, one of them at least!

Marek, my magnificent mentor (adjectve chosen because of it’s alliteration with both Marek and mentor) was busy applying patches for me, and at long last, eclipse-wtp-webservices and eclipse-wtp-jeetools were buildable in F17, so I went through the motions to get them included.

Back to our fundly bundly jarry jars that are blocking JBoss Tools Archives. The remaining one was TrueZip. Upon investigation into this, I found that the bundled version (6.6) is obselete, and the current stable version (7.5.5) works quite differently. Hmm. I decided to go onto everyone’s favourite IRC channel, #fedora-java, and ask what the best course of action would be. I honestly didn’t want to package the old obselete version, but I also didn’t know if I could easily fit the new version in easily. My question was already being answered, or at least the concept was being discussed indirectly, when I hopped on to ask. I decided based on the conversation that was being held, that the only way forward would be with the up-to-date version of truezip, no matter how difficult it might be to get it to work with JBoss Tools. The last two F’s in the ‘Freedom, Friends, Features, First’ moniker, are very relevant in this situation!

I got to work with the packaging of that, and it turns out that it’s the largest spec I’ve written by far, currently standing at 432 lines. I ran into a few problems here and there, some of them were just me being stupid (this causes lots of problems always), the rest seemed quite soveable, just requiring a few patches here and there. TrueZip looks like a really useful idea, and it seems to be under heavy development for the last while, so that’s a plus! I found it a lot easier to package than the Eclipse stuff that I’d been working on for so long now, but it still took a while, because of the modular way in which it’s structured. Then again, maybe I found it easier because I’d learned so much by going through all that Eclipse stuff. I contacted Christian Schlichtherle, the lead developer for TrueZip, and he seemed quite enthusiastic about having it packaged in Fedora!

There’s a useful (although a little incomplete) page on the TrueZip site, on how to migrate from TrueZip 6 to TrueZip 7; so I’ve started following that and I’ve patched up the JBoss Tools stuff to work with version 7. The problem now is that I don’t know how to get Maven to find it, so I’m not sure if it works fully. I’m sure it’s an easy fix, and the wizards of the Fedora Java SIG will point me in the right direction. Unfotunately #fedora-java changes dramatically at the weekend: a hive of activity during the week, but a ghost town at the weekends!

This time next week, I hope to have the next iteration of the actual JBoss Spin available, hopefully it will finally have the JBoss Tools AS functionality (working with the packaged jboss-as). I realise that I’m way behind where I’d proposed that the Spin would be at this stage, but in hindsight I think my original expectations were a little unrealistic (for me at least)! It will be done, mo matter how long it will take!

Fedora JBoss Spin - GSOC 2012 update 5

25 June 2012

This is the fifth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

As I mentioned last week, first on the agenda this week was getting eclipse-wtp-jeetools in for review. I got that done, and it’s currently being reviewed.

Next was eclipse-wtp-jpa (dali). I had started a spec for this last week, but then I realised that I would need more features from it, so I tried building those. I eventually got that done, and the package is now ready for review also (although it depends on eclipse-wtp-jeetools, so it will have to wait until that is approved.

After those two packages were ready for review, I had to think about what the next step was, since those were the main focus of my attention for a while now. I went back to jbosstools, to see how far I could get with that, with the new packages. I tried building a few times, and with each error, I added in the missing stuff (pre-built) to the local repository, just to see how much more would be needed. After a few tries, I had enough to build the parent,common,usage,archives,jmx, and as components. Then something silly happened: my focus shifted away from this task, to other things (while I was waiting for the 3.3.0.Final release of JBoss Tools to download). When I got back to it, I forgot that I had included some prebuilt plugins in the local repository, remembered that the AS component was building, and I thought “Oh great! Now I can go and make a spec for that”. When I finally got the spec made, it took me a while to realise why the package wouldn’t build, and needless to say, I was a little annoyed when I figured it out!

Luckily, all the stuff that I was then missing was missing features from eclipse-wtp-sourceediting. That sounded fairly straightforward, just add the missing stuff, submit a patch, Bob’s you’re uncle, right? Wrong! The packages eclipse-wtp-webservices and eclipse-wtp-jeetools depend on the current feature that’s being built by eclipse-wtp-sourceediting, and the new features that would be added, depend on those packages! I was almost finished adding these features when I realised the potential problem. I asked in #fedora-devel to see if it was possible to have a situation where packages depend on each other, and I was told that it was ok, once it’s documented. To be honest, I’m not fully convinced though, it seems like a terrible idea!

Getting these features to build required quite a few patches to other existing packages, that didn’t have the right OSGI Manifest information. This seems to be a recurring theme. It might be just me, but it seems that the there is usually a huge gap between what OSGI information a package contains, and what eclipse/eclipse-pdebuild expects. This is quite frustrating, as it’s not always apparent why things aren’t building or working the way one expects them to. It’s not always apparent to me at least. All of these patches will delay things a bit, hopefully not too long.

After getting that all done, I went back to jbosstools again to see how much further I could get,with mixed results. It was working if I built it manually, but there were strange compilation errors from the spec, even though everything I was doing manually, was pasted from the spec file. It took me quite a while, but I eventually figured out that there are prebuilt jars in some of the components that I wasn’t removing when building manually. These of course were being removed in the spec file, so that’s what was causing the problem. Some of the prebuilt jars have equivalents provided in Fedora packages already, but of course, some of them don’t. That would be too easy! The new packages that need to be built have now been added to the list on the wiki page.

I was going to rant more here about the structure of eclipse webtools…the differing locations of plugins from the features that build/provide them; but it’s late, and I’ve already wasted enough energy today thinking about them!

Fedora JBoss Spin - GSOC 2012 update 4

18 June 2012

This is the fourth weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

This week I don’t have a detailed day-by-day breakdown of when I did everything, mainly because I can’t remember all of that!

As I mentioned last week, I finished by getting a subset of eclipse-wtp-jeetools working, but I was having trouble with one plugin of the org.eclipse.jst.enterprise_core.feature. At the time I wasn’t sure if I would need it immediately, but I didn’t want to submit for review until checking to see if it could be fixed. I sent a message to the java-devel mailing list to see if anyone could explain the problem. As I dug deeper back into jbosstools, I realised that I would need a lot more from both eclipse-wtp-webservices and eclipse-wtp-jeetools! Even though all of this webtools stuff may not be needed by Eclipse for the AS tools subset of jbosstools; to build the AS tools, other parts of jbosstools need to be built. It’s a similar process for almost all of the jbosstools: first build the target platform and the parent, the build the common component, then build any other components that your component depends upon (such as archives, tests, jmx, and usage; in the case of the AS component). This means that all of the heavier dependencies that are being packaged now, although they aren’t all necessarily going to be used by the AS component directly, they will hopefully pay of in the end, allowing us to package more of the other components more easily after.

Although I’d had a review request ready for eclipse-wtp-webservices for a while at this stage, it still relied on a few other bugs that I’ve mentioned before, being fixed. I had submitted patches for those, but then I forgot to follow up and make sure they were looked at. With the help of Stanislav Ochotnicky, Aleksandar Kurtakov and Mikolaj Izdebski, these got sorted out, and Mikolaj then helped me again by doing the review request. Thanks guys!

After I got eclipse-wtp-webservices into rawhide, I started looking back at that problematic eclipse-wtp-jeetools plugin that I mentioned above. It drove me to despair for quite a while, as there were no obvious tell-tale signs like before. It turns out that it was a problem with the OSGI manifest again! Even though eclipse-pdebuild seemed satisfied with the versions of the particular versions of the bundles that were being provided by the eclipse-wtp-webservices, they were failing silently, until the compiler couldn’t import them.

Once I figured that out, I tried to see if I could get more of the other features to build, starting with org.eclipse.jst.enterprise_ui.feature, since I would need it for the jbosstools common ui feature. This however would require the org.eclipse.wst.ws_ui.feature from webservices, so I got working on that first. At the moment, that’s now in the eclipse-wtp-webservices that’s in rawhide, and eclipse-wtp-jeetools will be ready for review tomorrow! :)

Next will be eclipse-wtp-dali, which I’ve also got a draft specfile for, but it needs some work yet!

Also, I wrote an additional blog post during the week, on how to create the Fedora JBoss Spin from the kickstart file that I created last week. I probably should have created the disk image and hosted it online somewhere, but unfortunately I don’t always have access to an internet connection where I can upload such a large file and still do other stuff at the same time! For the next milestone release, I’ll aim to get an image up!

Creating the Fedora JBoss Spin (GSOC 2012)

14 June 2012

In my last post, I mentioned that I had created a kickstart file to build what will become the Fedora JBoss Spin. Noting that this is still very much in development, and hasn’t been submitted to the Spins SIG yet, I even hesitate to call it a ‘Spin’ yet. Anyway, in this post I will briefly explain how to create an ISO disk image from that file. Once the ISO image is ready, it can be installed as a live media onto a USB stick or a DVD; or it can be run in a virtual setting.

Install spin related packages

The livecd-creator tool is provided by the livecd-tools package, and we also need some of the standard kickstart files from fedora-kickstarts:

sudo yum install livecd-tools fedora-kickstarts

Prepare for build

Next we need to get the custom kickstart file for the Fedora JBoss Spin:

curl -o fedora-livedvd-jboss.ks http://galileo.fedorapeople.org/fedora-livedvd-jboss/fedora-livedvd-jboss.ks

Before we can build it, we must make the standard kickstart files that we need, available in the same directory that we’ve got the custom one in, so in that directory:

ln -s /usr/share/spin-kickstarts/fedora-live-base.ks
ln -s /usr/share/spin-kickstarts/fedora-live-minimization.ks

Build!

Now the image can be created. Note that this step may take a while. It will also download a lot of packages, so the your internet connection will play a part in how long it will take:

sudo livecd-creator -v --config=fedora-livedvd-jboss.ks --releasever=17

What’s next?

Now that you have a freshly squeezed live ISO image, you can do what you want with it. You can use it in your favourite virtual environment, or you can put it on a USB stick and live boot from that!

Fedora JBoss Spin - GSOC 2012 update 3

11 June 2012

This is the third weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

As I mentioned last week, this week would see me inactive until later in the week. Unfortunately, I didn’t get back to normal until Friday morning, so here’s a short summary of what I’ve been doing since then.

Friday

When catching up with my emails, I got a message from rel-eng on the ticket I had submitted last week regarding the buildroot override: it’s now a self service procedure, so obviously wherever I read about it on the wiki is a little out of date! I created it, and then Mikolaj Izdebski helped me out a lot by reviewing the wsil4j package that I was waiting on. Again, this took a few iterations before it was in an acceptable state to be pass the review stage, and I learned more each time.

After that, I got started on making an initial version of the kickstart file for the spin, as this weekend is what was set for the alpha release. My first few attempts at this were unsuccessful, and I was getting not-altogether-helpful error messages.Everything would work fine, until I would add the Fedora Eclipse yum group. I had no real idea why it was failing here, as package groups are widely used in kickstart files. I didn’t get very far with this on Friday, as each try takes a significant amount of time and data!

Saturday

I continued with trying to get the eclipse group to work with the kickstart file, and eventually I made a breakthrough: I was referring to the package group the wrong way.With yum, there are two ways to install a package group. In the case of the Fedora Eclipse group, the most explicit way would be yum groups install Fedora\ Eclipse; with the alternative way being yum install @eclipse. In the kickstart file, I was trying a horrendous mix of the two: @Fedora Eclipse. I’m not exactly sure how to get the @name for package groups, usually I install by getting a list of all groups with yum groups list, but that doesn’t provide the short name. I’m not sure how I figured out @eclipse, it may have been a guess.

Since none of the JBoss Tools are ready yet, they obviously can’t be included in the spin, so for now it’s similar to the default Fedora live desktop spin with GNOME 3. The most notable differences are that it includes the Eclipse IDE, with the webtools packages that are packaged for Fedora thus far; and also JBoss AS 7. These are sizeable additions, so at around 1.3GB, the spin is already too large for a CD. The other changes that I’ve included are just to include some tools that I think could be helpful for a JBoss developer: git, svn, cvs, and mercurial, and xchat. I also modified the favourites menu, as can be seen in the screenshot below. I’ve put the kickstart file online here.

Sunday

Today I got back to trying to package the eclipse-wtp-jeetools that I had started. Thankfully, there are not as many patches needed for this as there were with eclipse-wtp-webservices! It mostly just needed some shuffling of the feature.xml file. There’s one plugin that is still proving problematic, so for the moment, it’s commented out, and not being built. I eventually got it to build anyway, and created a working spec file for the package. I won’t submit it for review yet until I communicate with some of the wise people of the Java SIG, and see if they have an solution to the issue regarding the plugin that refuses to build.

That’s all for this week, I’m sure the coming week will be a busy one!

Fedora JBoss Spin - GSOC 2012 update 2

03 June 2012

This is the second weekly installment of my GSOC blog series. More info on the progress of the Fedora JBoss Spin can be found on the Fedora wiki.

This week was quite productive I think! There were of course points at which I was tearing my hair out, but the Fedora Java SIG were always able to shed a light on my problems. Here’s a day-to-day breakdown of what I’ve been doing since the last update.

Monday

On Monday, I drew up spec files for my first two packages! uddi4j and wsil4j, are both relatively small packages, but they are required for eclipse-wtp-webservices. Marek Goldmann, my mentor, reviewed the first one, and by Wednesday we had a package of sufficient quality for Fedora! wsil4j depends on uddi4j, so it was necessary to wait a while before getting a review for it. I actually thought that uddi4j would need to be in fedora stable repos before a review could be done on a package that depends on it, but apparently not always! I filed a buildroot request with rel-eng today, so hopefully I can get it reviewed soon.

Tuesday

I installed the packages that I made on Monday, locally through yum, and tried again to build eclipse-wtp-webservices. I found another dependency that seemingly wasn’t being satisfied, javax.activation. I found a library called geronimo-activation and drew up a quick spec file, and sent it in for review. I later found out that that was in vain, as javax.activation is now provided by the jvm. After that, I was getting some quite strange problems (this was one of the moments that I was tearing my hair out!), and I had no idea what was going wrong. I spoke with some of the experts in #fedora-java on freenode, and it got me thinking that it’s possible that some other packages in fedora might provide what I’m looking for, but have problems. I found one of those problems, fixed and made patches for it, and submitted the patch to bugzilla, here.

Wednesday

On Wednesday, I found another stumbling block with an already-packaged-in-fedora software that I needed, so I patched that, and submitted the patch here. Then I finally got eclipse-wtp-webservices to a stage where it got to the compilation stage of the build. I was far from out of the woods here though, as there were many compilation errors. I started looking into these, and creating patches for them. The reason for most of the compilation problems, was that some of the libraries that the plugins were depending on, were old versions, and we’ve only got the ‘latest-and-greatest’ in Fedora, which have lots of additional abstract methods that need to be implemented, but weren’t being, in the plugins in question.

Thursday

The aforementioned uddi4j package from Monday was now in a condition that it could be submitted to the fedora repositories, so Marek helped me with all of the stages in getting it there. After that, I continued patching up the eclipse-wtp-webservices stuff. As this was different from just patching a spec file, or OSGi manifest for a fedora package, I decided to ask the fedora-devel mailing list for advice. I wasn’t really sure what to do, since after I would fix a number of errors, and try to rebuild, an even higher number of errors would appear this time (another one of the hair-tearing-out-of moments), and the number of patches was growing steadily. Having heavily patched stuff doesn’t seem to align very well with the Fedora way, of sticking closely to upstream. As it’s really my first time patching anything to this degree, I was unsure as to what to do. Maybe upstream were sticking to older versions of dependencies for good reasons? Maybe I was causing a lot of additional problems with my patches? The fact that I’m doing this as part of GSOC, where I’ve got some time constraints, I was also unsure whether I could wait for upstream to implement my patches so that I could make a fedora rpm. Thanks to Aleksandar Kurtakov (who is always very helpful), who replied quickly and put my mind at ease.

Friday

I found a bug in the OSGi manifest of the uddi4j package, so I fixed that and submitted it as an update. This again was a first, updating a package, and not having to submit it for review! I also eventually got to the end of the compilation errors in eclipse-wtp-webservices. I was quite surprised, I didn’t expect it to build. When I saw BUILD SUCCESSFUL, my first reaction was “This error message is different to the others”; then I thought that I must have done something wrong (which I guess is still entirely possible!

Saturday

I spent most of Saturday drawing up the spec file for eclipse-wtp-webservices. It was the largest one that I’ve done to date, with the most patches, so it took a while. I found it very useful to get ideas from the other eclipse-wtp-* packages that we already have in Fedora. By the end of the day, I had one that I was happy with. It was quite late though, so I thought it best to sleep, and have a quick look over it in the morning with fresher eyes.

Sunday

Today I fixed a couple of tidbits in the spec file from yesterday that I got from rpmlint. I submitted it for review, and I also installed it locally, so I could get on to the next step. I started looking at eclipse-wtp-jeetools, which is the reason I needed webservices in the first place. I’ve made a couple of changes, and I’m at the stage of compilation errors already (we seem to have all of the dependencies here in this case I think). I got a little panicky the first time it failed at compilation stage, because there were just shy of 3000 errors. I’ve managed to play around with the feature.xml, and now there are only 100 errors. Of course, it’s possible that once the 100 errors that are sitting on top of the previous 3000, and are just preventing them from happening yet. I wouldn’t be surprised if I saw those again, but I may cry.

Wow, this wasn’t supposed to be this long, but I guess it was a productive week, and I learned a lot. Next week’s report will be significantly shorter, I imagine, as I will be away for some of the week. I do hope to get an alpha kickstart file for the Fedora JBoss Spin though, to stick to the schedule! I’m aiming to have jboss-as and eclipse in the alpha release, I’m not sure how much else yet though!

Fedora JBoss Spin - GSOC 2012 update 1

27 May 2012

Google Summer of Code 2012 started last Monday, so this post is the first in a series of weekly updates that I will be posting here.

As I missed out on most of the earlier ‘get familiar with everything phase’ (university exams, and then a viral infection), I’ve been doing my best to get on track this week. At the suggestion of my great mentor Marek, I’ve created a wiki page on the fedora wiki to track progress of what is done and what is yet to do. For the moment, the focus is on packaging JBoss Tools. As expected, this will require additional dependencies to be packaged, and the dependencies may have their own dependencies that need to be packaged. What we know so far is listed on the wiki page.

The first thing I did then, was to create an initial spec file for what will be the eclipse-jbosstools package. Each plugin requires the parent and target platform to be built, so I managed to get that much done at least. It required some patching because of some dependencies that we don’t yet have in fedora, but it builds, so that’s good. As it was my first time creating a spec file, I got some feedback from Marek on it, and there were a few things to fix!

Next I tried to build the ‘common’ component, which I think will be a subpackage of eclipse-jbosstools, eclipse-jbosstools-common. This is the first major hurdle, as it needs eclipse-wtp-jeetools, which needs eclipse-wtp-webservices. Since Friday, I’ve been trying to build these, and I think I’m very close to getting eclipse-wtp-webservices working (just getting some compile errors now). The biggest problem here I think is that in Fedora 17, we’ve got Eclipse 4.2 which isn’t released yet. Anyway, hopefully tomorrow I can get some help with fixing that, get it packaged, and get it in for review.

This week I’ve learned to use some new tools and gotten more experience with tools I didn’t have much with; including eclipse-pdebuild, maven, and even svn and cvs . I also learned a lot about rpm spec files, mainly from downloading srpms, and seeing how other people have done stuff. I’ve also gotten quite frustrated a few times: A couple of times I spent time trying to figure out dependencies for different parts of the eclipse-wtp stuff that we need, then taking a wrong turn somewhere, and spending ages going into a dependency hole that wasn’t even necessary. Taking a break solved the problem a few times, I guess by forcing me to take a step back and see my mistake when I get back to it. Another frustration I’ve had is slow connection speeds when downloading some stuff from svn and cvs, but hopefully I won’t have that problem too often!

That’s all for this week, hopefully I’ll have even more to share next week!

Google Summer of Code 2012 - Fedora JBoss Spin

24 April 2012

So yesterday I found out that I got accepted into the Google Summer of Code program. It’s really exciting news for me, being enthusiastic about free software and open source, and also because I’ll be getting money to contribute to a project that I’ve been hoping to get more involved with anyway. I’ll be working with the Fedora project, to help with the packaging of JBoss software, and eventually I’ll create a JBoss spin, which will include JBoss Application Server 7, JBoss Tools for Eclipse, and JBoss Forge. I’m currently doing exams in university, so my activity on this will be limited until they’re finished. Then the hard work begins! I look forward to learning from all of the talented people who are already involved in the Fedora community. For more info on what I’ll be doing, the proposal is available on the wiki. I’ll update here with any interesting progress!

What ARE YOU looking for?

27 March 2012

Adjust screen brightness on fedora 16 - try this if it’s not working

28 November 2011

Just in case this can help anyone on the fedora planet or other, who doesn’t visit fedoraforum.org or ask.fedoraproject.org very often. Admittedly I don’t visit there very often myself. A quick search in bugzilla shows some possibly related bugs.

If you don’t seem to be able to alter the screen brightness in fedora, either with the Fn keys, or through the gnome-control-center, try reinstalling bash. I rarely have the screen brightness on full on my laptop, so when I couldn’t change it on f16, it was ever so slightly annoying! I honestly wouldn’t know what to file a bug report against, so I started searching. First stop was that shiny new ask.fedoraproject.org instance, which led me to this post on forums.fedoraforum.org

So with the Fn keys not working, which had always worked previously; and the slider in the screen settings in gnome-control-center having no effect, doing a simple sudo yum reinstall bash was the answer.

Update: It’s listed in the ‘Common F16 Bugs’ here:

https://fedoraproject.org/wiki/Common_F16_bugs#Bash_entry_in_.2Fetc.2Fshells_disappears_after_update

Add applications to autostart on login, in GNOME 3

14 May 2011

Since I started using GNOME 3 a while back, one feature that I’ve wanted to see is an option to have certain programs start up when I login. Unfortunately I haven’t had time to look into this until now, and a quick look in the release notes pointed me in the right direction. There are two ways of doing this; neither of which I find very obvious, and both do essentially the same thing. The easiest way is by using gnome-session-properties. Unfortunately, this is something that the shell in GNOME 3 doesn’t seem to know much about. Entering it in the search bar in the shell doesn’t provide anything useful, apart from a wikipedia or google search possibility. To open the dialog, we must either open a terminal and enter gnome-session-properties, or hit the trusty Alt+F2 and enter it there. At this point, I would like to point out that the Alt+F2 command launcher now has tab auto-completion (I’m not sure if that was a feature before or not, but I’ve only noticed it now)!

As you can see, this presents us with the familiar Startup Applications Preferences dialog, where we can add/remove applications to our hearts content! As you can see from the screenshot, the first application I added was RSIBreak. This is a handy little utility to help prevent Repetitive Strain Injury.

The alternative way to add something to startup on login is to manually create the.desktop files in ~/.config/autostart. Some examples can be found in /usr/share/applications.

Older posts are available in the archive.

Gerard Ryan's Blog

Using gpg-agent with Emacs flatpak

Bonus Emacs content!

Testing Jenkins Pipeline shared libraries

The library definition in the global Jenkins configuration

The Jenkinsfile in the library repository

Pre-merge testing against other jobs

Trusting Jenkins Pipelines in public repositories

My Podcast list in 2016 🎧

Software development

The Changelog

Ruby Rogues

Software Engineering Radio

NodeUp

FLOSS Weekly

The InfoQ Podcast

Thoughtworks Beacon Podcast

Free software / Free culture

Linux Voice

Bad Voltage

Hacker Public Radio

Free as in Freedom

General Interest

Freakonomics Radio

Star Talk

Ted Talks (audio)

Two Pump.io client apps for Fedora (COPRs)

Dianara

Pumpa

Prank - alias sudo in BASH (or why you should start escaping privilege-escalating commands)

Fedora Java Spin - GSOC 2012 update 13 (Final)

Fedora Java Spin - GSOC 2012 update 12

Fedora Java Spin - GSOC 2012 update 11

Fedora JBoss(?) Spin - GSOC 2012 update 10

Fedora JBoss Spin - GSOC 2012 update 9

Fedora JBoss Spin - GSOC 2012 update 8

Fedora JBoss Spin - GSOC 2012 update 7

Fedora JBoss Spin - GSOC 2012 update 6

Fedora JBoss Spin - GSOC 2012 update 5

Fedora JBoss Spin - GSOC 2012 update 4

Creating the Fedora JBoss Spin (GSOC 2012)

Fedora JBoss Spin - GSOC 2012 update 3

Fedora JBoss Spin - GSOC 2012 update 2

Fedora JBoss Spin - GSOC 2012 update 1

Google Summer of Code 2012 - Fedora JBoss Spin

What ARE YOU looking for?

Adjust screen brightness on fedora 16 - try this if it’s not working

Add applications to autostart on login, in GNOME 3