prose :: and :: conz


OSS Scala starter kit

In my recent post on Java properties in Scala, I decided to flesh out the idea more fully as an open-source project. While it is not the first open-source Scala project I’ve started (see SNMP4S), there were several things I had already forgotten about which help make a good project to share with the community. In this blog post, I’ll highlight what I’ve learned which I think are good for creating an open-source Scala project. I am working with the assumption you know very little about open source development in general, so skip the headings which don’t apply. I am also assuming you’ve already researched your OSS idea to see if someone else hasn’t already begun a similar effort which you could join. My hope is folks who have been getting comfortable with Scala, perhaps those of us among the legions having gone through the Coursera offerings, can be unleashed to contribute their great work to the community.

Prerequisites

The first thing to do is buy a domain name if you don’t already have one. Personally, I bought my twitter handle as a domain so I could use it for my code. This will become your package prefix and gives you an organization for publishing your artifacts. What you come up with here isn’t very important, it just needs to be a domain you legitimately own.

You’re welcome to keep your code where ever you like, but most Scala projects are hosted on GitHub. Even Scala itself lives there. So sign up for an account, because we’ll be using it soon.

If you’re sentenced to Windows, you’ll need to get git. I recommend Git Bash. Cygwin with git is an option too. I’m hopeful that you are already reasonably familiar with git as an introduction to that would be an entire post in of itself.

Bootstrap the project

In GitHub, create a new repo. Enter a name and description, but don’t bother with a README or license. Clone this project to your machine.

To make our lives easier, I’ve set up a giter8 template. Even if you are already working with a code base, this is a good place to start because the template includes some plugins and sbt settings which you will need later in this tutorial. Follow the steps to install giter8 (NOTE: if you already have g8, make sure you update to at least 0.6.1 for this template). Part of this install is to first install Conscript, and we Windows users have to go here to get the installer. Once you have conscript and g8 in place, see the readme on my g8 project for details on generating the project. Run g8 in the same directory you executed the git clone (one directory up from the root of the project), and when prompted use the same name you gave the project in GitHub.

Scala Versions

After you’ve generated the project template, take a look at the build.sbt file. Take notice that I’ve set this to an older version of Scala, namely 2.9.1. Then just below the version setting, you’ll notice crossScalaVersions. This allows us to build our project for multiple versions of Scala. Historically Scala isn’t compatible across binary versions, so code compiled with Scala 2.9.1 cannot run with the Scala 2.9.2 library binaries, for instance. This has gotten a little better as now any version in 2.10.x is binary compatible, hence you will only see the latest 2.10.3. To reach the broadest audience, you should aim for the oldest version you can support.

You can take advantage of the cross-versioning simply by prepending a + in front of targets you want to run across all versions. For instance, you can run your test code against all versions of Scala with the command +test. Later on when we are ready to publish, we will use +publishSigned

Not only are we explicitly setting the Scala version, you will also notice that there is a project/build.properties for the sake of making the sbt version explicit. This will ensure that folks who fork your repo will not encounter errors due to running a different sbt version than you intended.

Readme

Next topic is the README file, arguably the most important piece of documentation a project has. Any good open source project has a good README page. The g8 template has already given you a README.md to start with. In particular for Scala projects, you’ll need to tell the users how to add the code to a project. This is covered in the Configuration section of the generated README and should be ready to go. It is a very good practice to keep the version up to date as you release new versions of your project.

I have a few other sections in the README which will likely be handy for any project. Do your best to add any useful documentation on this page. The quality of an OSS project is greatly impacted by the quality of the README. Other sections to think about are Community (where will users find help?), Contributions (how can other developers contribute?), etc.

Scaladocs

The second most important documentation you can give your users is the Scaladocs. We can also publish this to GitHub as one of their Pages.

First you need to follow the steps in Creating gh-pages branch from the sbt-ghpages project site. Be sure to create a temporary directory as Josh suggested in those steps. Otherwise you’ll wipe your project. After pushing the new branch, give GitHub about 10-15 minutes to have your site available. It will be located at http://github_id.github.io/project_name, substituting github_id and project_name accordingly.

If you are a unix user, then publishing your docs is super easy. You just push your project with the ghpagesPushSite task.

If you are a windows user, you’ve got to do this a bit more manually because git is not readily accessible. Run the doc task and manually copy the target/scala-2.9.1/api/ directory over to the temporary directory you created earlier for creating the gh-pages branch. Copy it to a sub-directory named latest/api/ to match where the plugin would have put it for you. Commit and push it to the gh-pages branch.

Once this is all done, your Scaladoc API link in your README should be functional.

OSGi

I’ve not seen much usage of OSGi in the Scala community, but it is easy enough to provide. In a nutshell, to be OSGi-compatible we just need to sprinkle in some useful information into our jar manifest file. This generated project should be able to handle it pretty well for you without any tweaking. I haven’t used an OSGi container in about a year, and I haven’t tested the bundles this project produces.

If you’re interested, run the osgiBundle task and examine the META-INF/MANIFEST.MF. You’ll see that we’re providing some information so that the jar file can be loaded in an OSGi container as a “bundle” where the dependencies and such are well-known. Note that this osgiBundle task does the same as package, except that it adds this nice manifest. With tasks like publish, the OSGi bundle task is used to produce the artifacts.

Publishing to Sonatype

Let’s now take a look at how to publish to Sonatype. Many mature Scala projects have long been published to Sonatype, so it will be easiest for our users if they are not required to add another resolver. Lately there have been easier publishing solutions available such as Bintray which the Scala community seems to be moving towards, tho. As you are about to discover, there are some initial steps and some overhead associated with each release which may be overkill for your small OSS project. Nonetheless, I personally go through the trouble to make it easiest for my users to utilize my libraries.

The first step is to signup at Sonatype. Then you will create a ticket for publishing rights for your domain name. They only handle these tickets throughout the work week, so if you’re doing this over the weekend you’ll be waiting until Monday to get your publishing rights. They’re quite responsive though. Once someone (typically Joel Orlina) grants you write privileges for your domain you will be ready to publish artifacts. In the meantime, proceed with setting up a PGP signature, configuring your Sonatype credentials, and writing code for your project.

Now let’s establish a PGP signature. The generated project is already configured with the sbt-pgp plugin and this documentation should tell you all you need to know to get this to work. Once you’ve generated a public key, you can export it per the aforementioned documentation or upload it manually somewhere like the MIT PGP Public Key Server. Keep the private key somewhere safe where you won’t lose it, like Dropbox.

Not only should your artifacts be signed, but you also need to provide your Sonatype credentials to upload artifacts. Your credentials need to be defined in an sbt file, but you obviously don’t want those included in your project and committed to a public source repository. Fortunately we can define them in an sbt file in ~/.sbt/0.13/plugins (Windows users, this will be your C:\Users\username\ directory). Create a file named credentials.sbt and add these contents…

credentials ++= Seq(
  Credentials("Sonatype Nexus Repository Manager",
              "oss.sonatype.org",
              "username",
              "password")
)

…substituting username and password accordingly.

All of the other prerequisites/requirements for publishing artifacts to Sonatype are handled by the sbt-pgp plugin and the generated project. Once you have your write permissions to your Sonatype repos, you can begin publishing. All you have to do is execute the task +publishSigned. This will prompt you for your PGP signature password, then begin uploading all of your artifacts to Sonatype.

Once your artifacts are uploaded to Sonatype, you have to release them. These are the steps I was referring to earlier that are a bit of overhead and likely overkill for your project. In a nutshell, you will close the staging repository that publishSigned created and release the closed repository.

Now set up a quick test project to see if you can download your artifact from Sonatype. Note that it may take a little time for everything to sync up and make the artifacts available. Also be sure to clear your Ivy cache and local repos to ensure that the public one does indeed get downloaded.

Release tags

This section added on July 27, 2014
Each time you cut a release, it is best to tag the code in git. This not only helps you keep a history of your work, GitHub lets you have an ATOM feed for tags. This is a great way for your users to subscribe to notifications of when your library has versions released. The feed URL format is https://github.com/{username}/{project}/tags.atom. For instance, the atom feed for lift-ng is https://github.com/joescii/lift-ng/tags.atom.

Tagging your code at the end of a release is a simple git command:

git tag -a <version> -m "<message>"

Where version is the version of the code you are releasing (without surrounding quotes) and message is just whatever message you’d like to add about this commit.

When you push to origin, you will need to add the --tags switch:

git push origin master --tags 

Publicizing

This section added on July 26, 2014
So how does the world know about your code? One great way to publicize your Scala OSS project is via implicit.ly. This creation (also from Nathan Hamblen who made the aforementioned giter8) is a cinch to use for your project. The giter8 template has already dropped off a notes/ directory to get your project ready for publicizing. Just be sure to edit both files according to your project before publicizing. Go to n8han/herald on GitHub for all the details you’ll need to get the word out. Once you are set up, you will be able to announce each completed release of your code on this blog.

Another great service put together by Nathan and friends is the ls.implicit.ly card catalog for Scala libraries. Here visitors can search for Scala libraries by name, tag, etc. Again, the giter8 template has already wired this service up for your project. Just go to the bottom of the build.sbt file to find the LsKeys to configure them exactly for your project, particularly the tags and description. Just as with Herald, you can update the card catalog with each release of your library.

That’s it!

You should be up and running now. I hope someone else finds this post useful. I know I’ll be coming back to my g8 template often in the future, and probably visiting this post to remember how I did all of this. Got some tips yourself for our budding OSS developers? Add them to the comments below.

Now go out and build awesome stuff!


Olde Comments
Tagged with: scala (41), tutorial (5), giter8 (1), open-source (1), sbt (1)