Pete and Jono had a Kernel Q&A on #ubuntu-meeting. Since the IRC logs are not so readable and split across pages, I would like to turn it into a readable format.

Text in [] Includes my comments or links

UndiFineD: What is the impact on performance on the BLK removal?

Should be a positive impact, and will make things more maintainable long term, will be more noticeable on multi-core boxes

kim0: Does Ubuntu server have certain tweaks applied to its kernel vs desktop

The only real tweaks are a few config settings and the increase on the number of CPUs the kernel supports and the default IO scheduler is Deadline

OkDucky: Why does the performance of ubuntu (10.10) reduce a fair bit when under heavy hdd load? is there work being done to improve this? (it could just be my hardware of course..)

 The IO Scheduler for 2.6.38 has been revamped so you should start to see better performance

Taek: How exactly does the glorified 200 lines kernel patch will benefit in 11.04

That is the auto cgroup stuff, isolate session tasks, it helps balance the load and will help increase performance. Bottom line is you will see a benefit on heavy IO

m4n1sh: What about btrfs? When is it going to be in a usable situation fit for default install?

Post 11.04 we are expecting it to shore up to be a default, right now its in the installer but you have to manually select it

sebsebseb: What are the main features of the next kernel?

Look at kernelnewbies.org for the 2.6.38 kernel, way to many to list here [*]

Also look at the A2 release notes some more info there as well.

Jay-Cee: Any improvements with btrfs performance?

File deletion is noticeable faster, but as with any dev system things are in flux

Tach: Will there be any fixes with the issues with input of some joysticks being detected as mouse input. An example of such an issue is here

Not sure this is an edev issue with X, not kernel per se

m4n1sh: Lennart posted a 2 line solution for the glorified 200 lines patch for kernel. Was Lennart’s solution better than that of kernel?

Lennart’s solution had a limited use case and was not general purpose unfortunately

m4n1sh: I heard that installing ubuntu using btrfs as FS takes some hours to install. Is it true? Any special reason for this problem (provided it is true)

That used to be the case, more recent versions of the kernel have fixed that specific issue afaik

kim0: How does the server’s kernel compare to other enterprise distros’ kernels , i.e. does the team regularly sync drivers for Enterprise grade cards (SAS, NICs, LOMs…)

So this is a multipart answer so bear with me while I type it out. We have a pkg call LBM that get updates on a few drivers (LBM == Linux Backport Modules). We also have for LTS releases backported kernels that have the latest in upstream hardware enablement. The kernels are call LTS Backport Kernels and are basically the latest release kernel running on the last LTS. Keep in mind this is NOT A DESKTOP feature. It is only supported on the server flavor. So yes we have some level between the two to refresh drivers for servers. 

m4n1sh: The problems with suspend/resume/hibernate lies in which part of the kernel? Is the situation improving?

We have a tool call fwts (in universe) that will test your bios and tell you what is broken. Run that if you are having issues then attach the out put when you file a bug. It will help us determine if its a true kernel issue (rare) or a broken BIOS

doctormo: What should you do if a kernel bug makes ubuntu unstable for a target machine? downgrade kernel or downgrade os?

Downgrade the kernel, way easier

Taek: Is there plans or even talk about having a system update without having to restart or install third party software such as ksplice.

The kernel will support kplice however we have no plans to do such a thing, there is already a commercial venture. That does that.

doctormo: Is there any plan to help move packages from module-assistant to dkms in debian/ubuntu?

There are no plans to do so, dkms is our preferred method for out of tree drivers

GauravButola: There are wirless modems (wireless broadband data cards) and TV tuner cards, which do not get detected by Ubuntu at all. What can I expect in 11.04 in this respect.

The biggest reason is that the firmware is not available or there is not a driver. If the firmware is available and we don’t have it file a bug and we will see what we can do to get it into the distro. As far as what to expect, there is better support based on bugs and user feedback.

doctormo: Does hard coding usb and pci ids into the kernel driver code make enabeling hardware that would work, but for a new id, more difficult?

It is harder but its the only way we have right now, device tree should fix this problem in the future but its not ready for prime time on X86

sebsebseb: I don’t really know much about USB 3.0, but when is Ubuntu likely to support it?

It entered the kernel in 2.6.35 iirc and has been in ubuntu. Older ubuntu kernels will get support as it bubbles into the stable tree from upstream

doctormo: Are there any automated kernel speed tests happening in order to improve perforance in the kernel?

Not currently, there are plans but that is more of a QA question (hint ask marjo on his turn). However there are other doing it

sebsebseb: Do you think Ubuntu 11.10 will use BTRFS by default, if so why?

At this point we have no idea, that will be determined based on stability at UDS

The latest discussion about Application Review Board has been very much insightful. The discussion took place on ubuntu-devel mailing list. Here are a bit of extra information which I gained from the discussion. This information is purely based on what people posted there and not my personal opinions.

The original post by Jono is here. Use Next and Previous to navigate.

The Pain point

Till now getting you application in the repositories was a tedious work. Most of the community packages reside in Universe repository. What if some application developer wants to target a version of Ubuntu which has already been released. Currenlty the only way is via third party repositories or via Launchpad PPA(even PPAs are third party repositories). Adding a new repository is not an easy work for normal users who just want things to work.

Incentive

This new process will be easier to market for the developers who want to target this platform. The developer spends most of his time developing the application rather than spending time on getting his/word work to the repositories.

Restrictions and Limitations

Applications for which the underlying libraries needs to be changed are not eligible for inclusion. During UDS these applications were called the “lead node” applications

Also it would be tough to review medium and large size applications. Simple applications can get through the process in less time.

These apps are not a part of Ubuntu and not a part of Official repositories. This is a method to get third party apps available in Ubuntu with least headache.

Packaging Responsibility

It’s the developer’s responsibility to get the apps packaged for each release and submit it for review. Every release won’t pick up these package from the previous release to avoid “dead package” dumping.

The application developer needs to package his application in a PPA, which will be tested and reviewed by the board. When passed it would be added in the Software Center. Using Quickly, it is even easier to get the apps in the PPA

Backports

This is different from backports since the definition of “backports” means that the package is available for next development release which is then made available in the current release.

Evan started a proposal on area51 of stackexchange for a proposal for “Ubuntu specific stackexchange”. Those of who you don’t have any idea about stackexchange, it is the same platform on which StackOverflow runs on.

I know there is a sharp line between people who think that we don’t need ubuntu specific stackexchange and those ubuntu needs it. The discussions are basically related to:

1. It should not be ubuntu specific , but Linux specific
2. Ubuntu already has a wealth of such Q&A and forums

Big people like Miguel de Icaza have given their opinion on why first is not a good idea. I respect their opinions, but it is the general public who decides whether it should be there or not

Now I would like to talk on second point

Ubuntu Forums

I know UbuntuForums is probably one of the biggest distro specific forum. The amount of activity and number of users are overwhelming. It was designed for discussion and mostly focused on follow-ups. One person can post multiple replies which make the whole thread reach insane levels.

Point no 2, Ubuntuforums is not a Q&A platform.

Launchpad Answers

This is probably the most competitive alternative to stackexchange as it is meant for answering questions. I can point out many problems in this due to which it is not so famous. Probably Launchpad team is more busy working Soyez and Rosetta and don’t have much time for Answers. I don’t blame them. Launchpad is a fantastic platform. The problems with current Answers are

1. Doesn’t show the picture of the person next to the answer. Due to this, I need to look at the name and try to recall. Profile pictures are more powerful than names as identities.
2. It does not allow favouriting/bookmarking a question. You can subscribe to the question,but not bookmark. StackExchange allows these two functionality separately.
3. You cannot Upvote/Downvote answers. This makes it pretty tough to make out whether an answer is helpful or not
4. Does LP Answers allow tags on a question? I was unable to find it. StackExchange allows it and also to search by tags. To make it sweeter, it allows only 5 tags on each question to avoid tag misuse.
5. Stackexchange asks you to edit your original answer instead of putting one more answer on a question. Makes sense! You don’t want hundreds of follow ups. All your answer are together. You can edit your answer anytime.
6. I can go on and on. The best thing should win. We should not show any favour. StackExchange clearly stands out. Now it’s time for Launchpad Team to get Answers in right state.

Leonard Richardson (the Launchpad rockstar) needs to read this post.

Before going ahead, please bookmark MOTU - Getting started and for more questions you can visit #ubuntu-motu and #ubuntu-packaging

augdawg: fingerprint authent. on lanchpad wont work

dholbach: please follow https://help.launchpad.net/YourAccount/ImportingYourPGPKey very closely and try again or talk to fine people at #launchpad

devildante: Will we fix real bugs?

dholbach: yes

There is a very real sort of bugs, FBTFS bugs and with that a new acronym, FTBFS means Fails To Build From Source

If a package doesn’t build, it can be because all kinds of crazy things. Sometimes the source code is actually broken, sometimes it’s because we lack the right version of some development library etc. It’s something you’ll probably struggle with a couple of times as an aspiring developer :)

So let’s head to http://qa.ubuntuwire.org/ftbfs/ - it shows a list of packages that FTBFS (yes it’s used as a verb) right now. Let’s first have a look at http://launchpadlibrarian.net/50406598/buildlog_ubuntu-maverick-i386.pykickstart_1.74-1_FAILEDTOBUILD.txt.gz which is the compressed log of the build on launchpad

So what you just saw when you ran pbuilder, just on lots of machines in Launchpad. First part is the installation of necessary packages, last part is the deinstallation of packages

So close to the end of the log you’ll find this

dh_usrlocal

dh_usrlocal: debian/python-pykickstart/usr/local/bin/ksvalidator is not a directory

dh_usrlocal: debian/python-pykickstart/usr/local/bin/ksflatten is not a directory

dh_usrlocal: debian/python-pykickstart/usr/local/bin/ksverdiff is not a directory

rmdir: failed to remove `debian/python-pykickstart/usr/local/bin’: Directory not empty

dh_usrlocal: rmdir debian/python-pykickstart/usr/local/bin returned exit code 1

So something with files being installed in usr/local as the  packagers, don’t do usr/local - it’s against the rules. usr/local is just for stuff the user installs manually. You can find more about what goes where in the FHS (file hierarchy standard) and the debian policy document.

“upstreams” are the software authors of packages included in ubuntu, they often release their “upstream source” on their own website, it often gets included in debian, where patches are added to make the packages build the debian (and ubuntu) way, and we often inherit the code just like that. The more upstream and debian and ubuntu are in sync the better. Sometimes we make different decisions, but those should be for very good reasons because every deviation means additional work when merging changes later on again. So, when I look at a bug like that I very often check “was it fixed in Debian or upstream already”?

So the package is called pykickstart and let’s check out https://launchpad.net/ubuntu/+source/pykickstart as you can see, it was introduced in maverick and is at version 1.74-1 right now

On the Debian side, we see similar information at http://packages.debian.org/src:pykickstart you can see that ‘sid’, the Debian development version, has 1.75-1 already. Click on the ‘sid’ link to get to http://packages.debian.org/source/sid/pykickstart and on the right side it has a link to the debian changelog. So this is a regular debian/ubuntu changelog entry that states what in this package upload was changed. Ss you can see, the maintainer wrote something about a new upstream version and “   * Update debian/rules: pass —buildsystem=python_distutils to avoid ftbfs.” “avoid ftbfs” is like music to my ears - to yours too?

We should get the Debian source and try to build it and see if that works. if you go back to http://packages.debian.org/source/sid/pykickstart - you’ll see a link to a .dsc file http://ftp.de.debian.org/debian/pool/main/p/pykickstart/pykickstart_1.75-1.dsc

When we installed the devscripts package earlier, we got a tool called dget which we’ll use that now to download the source from debian as

dget -xu http://ftp.de.debian.org/debian/pool/main/p/pykickstart/pykickstart_1.75-1.dsc

To build it, you know already what to do:

sudo pbuilder build pykickstart_1.75-1.dsc

this will take a little bit now, so let’s all try to be patient and see how it works out

Questions

Noz3001:  Why -“xu”?

dholbach: without -x dget will not unpack the source (you’re right, we could have ignored that - usually it’s very helpful to get the source unpacked immediately to dive right into it) and without -u dget will complain about a missing gpg key, because it can’t confirm the identity of the uploader (you probably won’t have the debian maintainer’s public key in your keyring)

umutuygar: umut@umut-laptop:~/code/ubuntu-classroom$ sudo pbuilder build pykickstart_1.75.dsc

dholbach: did you execute the dget command above in the same directory?

ElPasmo: How you get a sponsor?

dholbach:  we’ll get to that in a bit, https://wiki.ubuntu.com/SponsorshipProcess if you’re impatient :)

chilicuil: can I run 2 or more pbuilder instances at the same time?

dholbach: yes, you can use pbuilder-dist (in the ubuntu-dev-tools package) for that and also there’s https://wiki.ubuntu.com/PbuilderHowto

penguin42: We’re past the point of pulling in debian packages for Maverick aren’t we? But if a package ftbfs and the debian one fixes it what happens?

dholbach:  I’ll get to that in a sec

Krysis:  how to add gpg into keyring?

dholbach: gpg —recv-keys <KEY ID>

umang: What’s the easiest way to find a nice diff of the two source packages (1.74-1 from ubuntu and 1.75-1 from debian)?

dholbach: nice question. You’d   apt-get source pykickstart   to get the ubuntu version too and then run

debdiff pykickstart_1.74-1.dsc pykickstart_1.75-1.dsc

(probably pipe it through less, filterdiff, etc afterwards)

Alright, pykickstart 1.75-1 from Debian succeeded to build on this end, so what do we do?

Let me explain a bit how Ubuntu and Debian fit in to the big picture of the release schedule I described earlier up until Debian Import Freeze, we automatically sync source packages from Debian and build them in Launchpad, if:

1. The package in Ubuntu is not modified (usually obvious by reading the version number, 1.74-1 vs. 1.74-1ubuntu1)
2. If the package is in Debian main (so free software)

So how do we live in this world?

If we introduce a change in Ubuntu, say we go from 1.74-1 to 1.74-1ubuntu1, then Debian introduces 1.75-1 so we need to decide if we can overwrite our changes (we sync the source) or if we need to merge the source manually

It’s very tempting to say “ha, no we can sync from Debian” but it’s INCREDIBLY IMPORTANT to READ THE DIFF as just reading the changelog is not good enough

READ THE DIFF

and TEST

Questions

malcolmci: dholbach: who decides if we should sync with upstream debian or keep ubuntu modification, for example ?

dholbach: That’s the best possible question. That’s the responsibility you have as a developer. You need to make sure in the best way possible that the actual change can be implemented in the way you suggested. If you’re unsure, you can ask lots of other fine people in #ubuntu-devel or #ubuntu-motu or #ubuntu-packaging. Also the sponsoring process (somebody reviews your code or suggestion first) helps with that in the beginning. It’s just important that you do your best in making sure and testing.

malcolmci: ah, so the package maintainer decides?

dholbach: In Ubuntu we don’t have dedicated package maintainers, so we maintain the packages as a big team.

mickstephenson52: would the easiest way to test be toi just install the deb file from debian and see if it works?

dholbach: thanks for getting me back on track :)

So, we have 1.74-1 in maverick and we have 1.75-1 in Debian sid, which means that we did not introduce changes in Ubuntu, so we’re sure that we don’t have to merge things manually.

We just verified that the package builds again in the new version and we’re still before feature freeze in maverick, so requesting a sync (https://wiki.ubuntu.com/SyncRequestProcess) should be fine, once we’re sufficiently sure it’s all good

Installing the .deb and see if it work is a great way

AudioFranky: I would expect that for very essential packages, like glibc, there is more than a single person to decide whether to keep or update?

dholbach: yes, those decisions are made at UDS in a bigger group and usually there’s agreement between people in Ubuntu and Debian who work on the package. When I said that “we don’t have dedicated maintainers” I meant that we have no “big maintainer lock” (somebody OWNS the package), but indeed we have people with a special area of expertise and that’s respected

Alright, the path ahead seems clear: test the package, make sure it’s alright, then engage with the sync request process.

Jere’s some more food for though, you can try it out yourself:

• http://launchpadlibrarian.net/50981070/buildlog_ubuntu-maverick-i386.xdotool_1:2.20100602.2915-1_FAILEDTOBUILD.txt.gz
• http://launchpadlibrarian.net/50411959/buildlog_ubuntu-maverick-i386.weave_1.3-2_FAILEDTOBUILD.txt.gz

but as I said: be careful and talk to people on #ubuntu-motu or #ubuntu-packaging if you’re unsure. Being unsure is a good thing and knowing when to ask a sign of taking responsibility and that’s valued.

Alright, now I have another interesting case for us:

http://launchpadlibrarian.net/49981844/buildlog_ubuntu-maverick-i386.xpad_4.0-5_MANUALDEPWAIT.txt.gz

the xpad package fails to build too.

Towards the bottom of the log you can see this:

• libmagickcore-extra: missing
• libmagickcore-extra: does not exist

so this means that one of the packages that is specified as a requirement to build the xpad package is not there. Let’s get the source code and let’s try to fix it. This time we’ll do it differently:

bzr branch lp:ubuntu/xpad

this will not get us the source package (remember the .dsc, .orig.tar.gz and .diff.gz file) but a bazaar branch (revision control system) with all revisions in Ubuntu and Debian. This will take a little bit, but is worth it.

In the case of source packages you just get one revision, in a format that works with all the debian/ubuntu build tools, but we’re slowly moving towards a world where we work with upstreams and debian and distributed version control is just what’s easier to use. I’d like to show you how easy to use it is

Alright, once bzr is done, run

cd xpad

and

less debian/control

Let me explain a bit what this is about:

debian/control contains information about the source package and the resulting .deb (binary) packages. In this case it’s relatively simple, let’s go through the source stanza first.

• Source: specifies the name
• Section and Priority are used by the package management to have a bit of metadata
• Maintainer is the maintainer in Debian
• Build-Depends is just what we’re looking for and this is the minimal list of packages necessary to build the package
• Standards-Version is the version of the debian policy this complies with and homepage just more metadata

This is all additional info regarding the source: The resulting xpad package is defined in the next stanza

• Package: is the package name
• “Architecture: any” means that the package needs to get built on every individual architecture available separately
• Depends: is the list of dependencies of the resulting package
• shlibs:Depends is the list of library packages that contain libraries the binaries in the deb packages are linked against
• misc:Depends a few additional Depends that might be useful

architectures are i386, amd64, powerpc, sparc, etc.

If you just put a bunch of python scripts into a package, or just some documentation that is architecture independent, you use “Architecture: all”.

All of these complicated computations are done by scripts in the debhelper package.

Description too is used by the package management tools

So debian/control is a file that contains lots of the definition of the package itself. You can check out the other files in debian/ on your own later on, https://wiki.ubuntu.com/PackagingGuide will help you with that.

bcurtiswx: will we ever mess with the bottom section?

dholbach: we could, for example if a user tells us that there’s a dependency missing or there’s a typo in the package description or something

So to avoid confusion: this example will just work in maverick, it’s a maverick build failure. If you don’t run maverick right now, that’s no big deal, just trust me blindly then.

http://launchpadlibrarian.net/49981844/buildlog_ubuntu-maverick-i386.xpad_4.0-5_MANUALDEPWAIT.txt.gz mentioned that libmagickcore-extra was missing

apt-cache search libmagickcore-extra (on maverick) will reveal that it’s now called libmagickcore3-extra

BeardyGnome13: i guess it’s best to run maverick on a separate machine / VM then?

dholbach: https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases has all that info

Ok, so we just change libmagickcore-extra to libmagickcore3-extra in debian/control and save the file. If you exit your editor and run    bzr diff    you’ll see your change. Now please run update-maintainer (script from ubuntu-dev-tools) to update the maintainer field which we were asked to do by our friends at debian to indicate that we made changes in the package)

now please run

dch -i

Also a devscripts tool which deals with all things related to debian/changelog it will automatically fill out some stuff for you. You just need to enter a descriptive message for your changes. I’d go for something like

* debian/control: updated build-depends from libmagickcore-extra to libmagickcore3-extra to fix FTBFS.

It’s important to note what you changed in which files and the more explicit you are about your changes, the easier it will be for you and others later on in understanding what you did. If you close a bug with your fix, please add   (LP: #123456). This special syntax will automatically close the bug once it gets merged in.

Now please save the file and run

bzr bd — -S -us -uc

This will build the source package from the branch. (-S -us -uc are arguments for debuild which is used internally) Then you move out of the directory and pbuilder the generated source package.

To test if that works now, this will naturally take a bit. Once you’re happy with all the  C A R E F U L   tests you did you can go back into the branch, run debcommit and then push the changes to launchpad and request a merge

https://wiki.ubuntu.com/Bugs/HowToFix explains the last bits and please check out https://wiki.ubuntu.com/MOTU/GettingStarted and join #ubuntu-motu and #ubuntu-packaging

As you can see, hugs are a vital ingredient of making Ubuntu ROCK :)

The first session of Ubuntu Develop week on “Getting started with Ubuntu Development” just ended. It was taken up by Daniel Holbach. I know reading IRC logs is boring, so here is sum up everything in human readable format. In between you will find quotation, which are actually when Daniel replies to some question asked by a person.

The entire text is given out by Daniel Holbach which I have converted from first person to third person speech.

IRC Channel: #ubuntu-classroom for classes (Freenode) and #ubuntu-classroom-chat for all discussions and asking questions

The whole session is of two parts

• Getting a couple of tools installed and the environment set-up
• Actively having a look at a few bugs and fixing them

Bookmark  MOTU Getting Started and Ubuntu Development: Using Development releases

It is necessary to get the latest development release. It needs not be installed as main system but can be on a Virtual Machine (example Virtualbox) so that we can have access to all the current development libraries, modules and stuff and you can test packages better

Now go and enabled “Source code” and “universe” in System → Software Sources → Ubuntu Software if you havn’t

Questions:

tillux: wouldn’t it make sense to do that now? because you need to download a lot of things/packages etc

dholbach: no, it’d take too much time. It’s fine - you can just copy the instructions later on

umutuygar: What programming language r u going to be using?

dholbach: we’ll see :-) I think for now I’ll stick to packaging basics where we fix a couple of bugs

BeardyGnome13: QUESTION: will this be GNOME-centric?

dholbach: no, not at all

Type this command in terminal

sudo apt-get install —no-install-recommends bzr-builddeb ubuntu-dev-tools fakeroot build-essential gnupg pbuilder debhelper

which will give you a bunch of tools that are going to be useful generally, not just in these examples

The explanation of the packages

• bzr-builddeb pulls in bzr which we’ll use to get the source code for one or two examples
• ubuntu-dev-tools pulls in devscripts which both are incredibly helpful at making repetitive packaging tasks easy
  fakeroot is needed by debuild (in devscripts) to mimic root privileges when installing files into a package
  build-essential pulls in lots of useful very basic build tools like gcc, make, etc
  gnupg is used to sign files in our case (uploads in the future)
  pbuilder is a build tool that builds source in a sane, clean and minimal environment it sets up itself
  debhelper contains scripts that automate lots of the build process in a package

Creating a GPG key

Now let’s move on the get our gpg key which we use it to sign files for upload but more generally it is used to sign and encrypt mails, files or text generally.

We use it to indicate that WE were the last to touch a file and not somebody else that ensures that only people who we know about get to upload packages

To generate a key, please run

 gpg —gen-key

sticking to the defaults is totally fine, for example don’t you need a comment. If you need more info on gpg keys, head to https://help.ubuntu.com/community/GnuPrivacyGuardHowto which talks about everything in more detail.

Enter your name, email address and just stick to the default values for now. It could be that gpg is still sitting there and waiting for more random data to generate your key - that’s expected and fine. Just open another terminal while we carry on, it’ll finish on its own.

If you have your own GPG key already, skip this step.

Setting up pbuilder

Please open an editor and edit the file ~/.pbuilderrc (create if you don’t have it yet). Please add the following content to the file

COMPONENTS=”main universe multiverse restricted”

and save it. Once you’re done, please run

sudo pbuilder create

More about pbuilder:

• It builds packages in a clean and minimal environment
• It keeps your system “clean” (so you don’t install millions of build dependencies on your own system)
• It makes sure the package builds in a minimal, unmodified environment
• Oo you ensure that the package does not just build because you made lots of changes on your system, but the build is reproducable
• You can update package lists (later on) with: sudo pbuilder update
• To build packages you run: sudo pbuilder build package_version.dsc

BeardyGnome13: will the transcript of the classroom be available later?

leak-BebeChooCho: Will there be full log available?

dholbach: Yes. It will be linked from https://wiki.ubuntu.com/UbuntuDeveloperWeek. For the impatient: http://irclogs.ubuntu.com

How pbuilder works:

• It first gets the minimal packages for a base system
• Stores them in a tarball
• whenever you build a package it’ll untar the base tarball, then install whatever your current build requires, then build it, then tear it all down again

somethinginteres: To confirm, the install of pbuilder etc should be done -right now- on our main box?

dholbach: It helps as you can perform the examples on your own machine and have it set up later on too

While gpg and pbuilder are doing their work, lets try out other tools

If you use the bash shell which is default, please edit  ~/.bashrc and at the end of it, please add something like

DEBFULLNAME=”Daniel Holbach”
DEBEMAIL=”daniel.holbach@ubuntu.com”

and save it. PLEASE USE YOUR OWN NAME, THANKS :-)

abhi_nav: I am doing everything in my main day to day usable machine. is it ok? I dont want anything to ruine my cute ubuntu. :)

dholbach: yes, it’ll be fine

once you’re done editing ~/.bashrc, please run   source ~/. bashrc  (it’s only needed once)

tpw_rules87: Should these match the values we put into GPG?

dholbach: yes

OK, with this out of the way, the packaging tools will know you by your name and you don’t need to enter it, for example if you do changelog entries, etc.

Questions:

augdawg: where is the .bashrc file located?

dholbach: ~/.bashrc , so /home/<your user name>/.bashrc

abhi_nav:do i need to write my that email which i was used to create gpgp key some months ago? now I have another email

dholbach: yes, it helps to have the same in there, but you can also add another user id to your gpg key later on. Refer to https://help.ubuntu.com/community/GnuPrivacyGuardHowto for that

malcolmci: is the preferred arch for dev’ing AMD64 or i386 ?

dholbach: every supported architecture of ubuntu will work fine

devildante: Will this session revolve around packaging bugs, or bugs in general?

dholbach: I picked a few packaging bugs, but maybe we’ll do something else later on, let’s see :)

augdawg: what does the ~ mean?

dholbach: my user name is “daniel” , so ~ for me will be expanded to /home/daniel

tillux: for those running on lucid: should pbuilder have been triggered with “—distribution maverick” ?

dholbach: for now that’s fine - if you set up a maverick chroot/virtual-machine/partition/machine later on, you can just repeat the steps. For this sessions it’s irrelevant.

BeardyGnome13: is my gpg key machine-specific?

dholbach: no, you can just copy it to another machine

somethinginteres: PBuilder is downloading for Lucid, should it be downloading for Mavrick or Lucid?

dholbach: see what I just said to tillux above

theneoindian: i hate to ask , but will pbuilder create eat up my b/w .. i’m on a limited b/w connection …

dholbach: you better stop it then and repeat later on on a different connection

By the time  pbuilders and gpgs are done, we keep talking a bit about ubuntu development and how it all works, we’ll make some good use of pbuilder and your shiny new gpg key later on :)

First things first: ubuntu is very special in how it’s produced and how we all work as you know it comes out every 6 months. That means we have a tight release schedule and everything we do and work on is defined by that schedule. Check out https://wiki.ubuntu.com/MaverickReleaseSchedule for the current release schedule for maverick.

The quick version: green means: lots is allowed here, red means: almost nothing is allowed here :)

Long version:

• Toolchain is uploaded for the new release (gcc, binutils, libc, etc.), so the most basic build tools are there
• New changes that happened in the meantime are synced or merged (more on that later on)
• Ubuntu developer summit (uds) happens where features are defined and talked about
• Up until debian import freeze we import source changes from debian semi-automatically
• Up until feature freeze we get new stuff in, work on features, try to make it all work
• If a feature is not half-way there yet by feature freeze, it will likely get deferred to the next release
• From feature freeze on you can see that lots of freezes are added throughout the weeks and you’ll need more and more freeze exceptions for big changes

The focus is clearly: testing, testing, testing and fixing, fixing, fixing.

The more obvious fixes are (a huge 20 million line patch is not obvious) the more obvious the better. The same goes for fixes that are introduced AFTER the release. We just want to fix stuff in …-updates if it’s REALLY URGENT stuff with really simple fixes. So as you can imagine, this means: introduce big changes early in the release cycle, so we can shake out problems over a longer time.

devildante: On the release schedule, I see a column named “Work Item Iteration”

dholbach: Ah, yes. If you check out http://people.canonical.com/~pitti/workitems/maverick/all-ubuntu-10.10.html you will see a graph and loads of text. This is a list of work items defined by specifications written at the ubuntu developer summit.  Work items are specific pieces of work that somebody at UDS decided to work on during the cycle. It’s more a tool for the managers at Canonical to keep their people busy. I mean… make sure we’re on track ;-)

Questions:

marceau: (I’ve asked this during the Userday as well but am wondering as to your input) Do you feel that the fast release cycle of Ubuntu might be hindering it’s progress? It seems a lot of time is spent in freezing and testing rather than working out new features.

dholbach: Not at all - I think it’s a good thing - it forces us to stay focused and make sure that stuff gets done. If we don’t get a feature done this cycle, it’ll be next cycle. Also it’s important for everybody else who works with the project, as it’s very easily predictable.

Daniel on how to get your stuff in:

When I spoke about gpg you noticed that I said that only people who “we know” get to upload packages directly. This means that as a new contributor you will have to work with sponsors who basically review your work and upload it for you. Once you did that a couple of times and they recognise you and your good work, you can apply for ubuntu developer membership and you can ask the people you’ve worked with for comments on your application.

It’s really not very complicated, you basically set up a wiki page with your contributions, ask for comments and submit for an irc meeting of the developer membership board and you’re done. No need to learn a secret handshake, send me money or anything else. It’s contributions and good work that counts. It helps if you’re a team player :)

Questions:

augdawg: what does an ubuntu developer membership get you?

dholbach: https://wiki.ubuntu.com/UbuntuDevelopers will tell you :). The most obvious thing is that you can upload changes yourself

prayii: But will sending you money help the process? *wink*

dholbach: unfortunately I’m not on the Developer Membership Board

tech2077: so when your a member do you get the ability to fix bugs somewhat more independently, trying to word it right

dholbach: It’s a question of trust: once you demonstrated that you do good work, you can get upload rights. Same goes for Ubuntu membership. People get an @ubuntu.com email address when they can be recognised for their good work in Ubuntu

marceau: what is the expected knowledge level of people taking part in this week’s sessions?

dholbach: I guess it will differ from session to session: in this session it’s good enough to be curious and have played with a terminal before and have a knack for making things work again :)

somethinginteres: is it the case that being an Ubuntu dev means being a programming vet or are there plenty of opportunity for n00bs to learn and contribute something..?

dholbach: there are lots of opportunities to learn, which is why we’re doing this whole thing :)

umang: If I’d like to contribute to both Ubuntu and Debian, which should I use?

dholbach: you can contribute to Debian by using Ubuntu and collaborating with Debian developers. It totally depends on you what you use and what you work on.

Now assuming pbuilder setup is complete, try getting the source of hello package

apt-get source hello

and run

sudo pbuilder build hello_*.dsc

this will kick off the build

abhi_nav: what .dsc stands for?

dholbach: great question. What “apt-get source” downloads is: (in most cases) an .orig.tar.gz file, which contains the source code of the software in a tarball released by the software authors. In some cases a .diff.gz file with changes that were applied to make the package build in the Ubuntu/Debian way and a .dsc file with metadata like checksums and the like it’s really not that important right now.

Once the build is done, you can check out the contents of /var/cache/pbuilder/result as it will contain the built hello .deb file :)

Now dealing with GPG. If you run

gpg —fingerprint your.name@email.com

It will print out something like:

pub   1024D/059DD5EB 2007-09-29

      Key fingerprint = 3E5C 0B3C 8987 79B9 A504  D7A5 463A E59D 059D D5EB

uid                  Daniel Holbach <dh@mailempfang.de>

…

In this case, the key ID is 059DD5EB , now please run

gpg —send-keys <KEY ID>

It will upload your _public_ portion of the key to keyservers which will sync the key among them. Once that’s done, you can tell Launchpad, which we use for all development about your shiny new gpg key - https://launchpad.net/people/+me/+editpgpkeys (you need a Launchpad account for this)

https://help.launchpad.net/YourAccount/ImportingYourPGPKey should help you if you run into any trouble

omid_: gpg: no keyserver known (use option —keyserver)

dholbach: try:     gpg —keyserver keyserver.ubuntu.com —send-keys <KEY ID>

TO BE CONTINUED IN PART II

Yesterday I was searching for a few fonts after being fascinated by Tumblr typography. I came across many excellent fonts at font squirrel too. I wanted a few monospaced fonts for coding which includes Consolas, Droid Mono etc. I knew that Consolas can’t be distributed freely, so I installed it manually. Hunting for Droid, I too managed to download and install it manually.

Now the questions is why I did it all manually? this is because Synaptic just lists down the fonts in a dumb way. I can’t find out new fonts etc. Software Center solves the problem by giving fonts a prominent section. Still it is not upto the mark. Look at the image below

Find something weird? Even though font’s have their own place, their categorization is pretty bad. All the fonts are just listed down. That’s it.

The fonts can have categorization as Languages based. Another way would be to have it categorized as Monospaced, Sans, Serif fonts. We can also have a section for recommended fonts like Droid Sans.

The league of Movable Type has a list of Open Source fonts. I tried finding them and failed to do so. Can’t they be added manually? Availability of good fonts is an important aspect for designers to look at Ubuntu/Linux.

To be frank, I suck at making mockups. I can make it on the board pretty easily, but on computer I might end up wasting a significant amount of time to draw them. I think the UI designers at Software Center would be more experienced in Mockups. Even they would know how it should look, so that it is in accordance with Software Center UI guidelines.

Lee Jones writes

Hi all,

  LP Bug #569882
    Suspend Lenovo Thinkpad X61 with card in SD card reader

I’ve been working on this bug on and off for ~4 weeks now. As it’s taken so long, I felt obliged to write a few words detailing the processes and methods used to debug this most horrific bug. The problem occurs when a laptop with an internal SD card reader is put to sleep (suspend or hibernate) when a card is inserted and mounted. Yesterday I discovered that this bug only affects cards which support an ext2 or ext4 filesystem (vfat and ext3 filesystems do not succumb).

This bug and others relating to it have been reported to Ubuntu over 130 times and ~20 bugs are related. A couple of those have an importance rating of ‘High’. Annoyingly, while I was working on this bug someone marked it as a duplicate of LP Bug #477106, so I hopped over and used that as the reference bug instead. It is there you can see the true range of this bug and where all other related bugs point. A list of related bugs can be found at the bottom of this page.

  LP Bug #477106 
    [regression] lucid alpha-2 and earlier freeze upon suspend with sd card plugged in with some hardware

The first debugging phase was to decipher which device was causing the issue. As the most common factor, I decided the best course of action was to hunt down the offending SD/MMC card reader driver and disable it. An ‘lsmod’ informed me that the sdhci and sdhci_pci drivers were in current operation. I ‘rmmod’ed them and attempted to sleep - good night ThinkPad. I had a dig around in the Power Management Utilities (pm-utils) and found the variable SUSPEND_MODULES which is sourced from the pm configs. I posted this on the bug report which seemed to please most:

  Momentary workaround:
    echo “SUSPEND_MODULES="sdhci sdhci_pci"” | sudo tee -a /etc/pm/config.d/suspend_modules 

Naturally, we’d like this bug to be eradicated forever and for Ubuntu to work seamlessly ‘out-of-the-box’. So I endeavored to find a more permanent solution within the kernel. After turning turning debug on within the sdhci driver and adding a few of my own DBGs, it was apparent that the system was coming to a screaming halt within the interrupt handler. My initial theory was that the SD card was being accessed following a full filesystem sync and free, thus causing a kernel oops or panic before the system could be fully suspended. I placed in some extra debug code to try and follow the thread of execution, but further attempts were seemingly futile.

(Thanks to cnd) I enlisted the help of ftrace to aid further kernel prodding, by enabling function_graph (Kernel hacking -> Tracers -> Kernel function tracer & kernel function graph tracer). Once enabled it can be controlled from within sysfs. To enable and disable, echo ‘function_tracer’ and ‘nop’ to /sys/kernel/debug/tracing/current_tracer respectively. Once enabled simply cat /sys/kernel/debug/tracing/trace_pipe, either to stdout or a file. This produce output similar to this:

 0)               |  schedule() {
 0)   0.334 us    |    rcu_sched_qs();
 0)   0.319 us    |    _spin_lock_irq();
 0)               |    put_prev_task_fair() {
 0)               |      update_curr() {
 0)   0.296 us    |        task_of();
 0)   0.951 us    |      }
 0)   1.584 us    |    }

The function tracer almost always ended in a spinlock, which lead me to my next theory, spinlock corruption. I enabled every configuration related to spinlocks and other locking mechanisms within the kernel. These included; Detect soft lockups, Detect hung tasks, Spinlock and rw-lock debugging, Mutex debugging, Lock debugging: detect incorrect freeing of live locks/prove locking and correctness and finally, Spinlock debugging. None of this helped.

Next I tried enabling the system console. Oddly, GDM decided it wanted to jump on the system console instead of it’s own tty. This made things a little more tricky. I killed it and tried to suspend manually using pm-suspend. A message informing me of the suspend would appear, then the console would go to sleep, rendering me blind as to what was happening. (thanks to cking for suggesting) CONFIG_PM_DISABLE_CONSOLE was the solution to this. It prevents the console from being allowed to sleep and subsequently turning off. After enabling this feature, it was plain to see, well… nothing! Even though I was on the system console, no kernel messages were produced. I knew they were there, as many of them had been saved in /var/log/kern.log and /var/log/messages, but they weren’t being displayed. I believe this has something to do with gdm and the differences in suspend regimes. I tried displaying pm-suspend using set +x. The script was displayed and nothing else.

Next on the agenda was to enable system console via serial. As I don’t own a ThinkPad dock, I had to use a serial->USB converter. Once up and running it was clear that the thread of execution was going a great deal further than the sdhci interrupt handler. As it turns out, the debug messages were finishing there because the final sync to disk was already performed. The drives (and therefore the logging) were no longer available. It is worth mentioning that suspend/resume logs are written on the resume, hence if suspend fails, the logs are never written - as with this case.

Now we have serial working, I figured it might be work giving kgdb a shot. I enabled it in Kernel hacking and tried to configure it over serial. The arguments kgdbwait and kgdboc=ttyUSB0,115200 were placed onto the kernel commandline, but kgdb just spat out “kgdb: Unregistered I/O driver kgdbts, debugger disabled.” I then tried to place them into manually via sysfs, giving: “echo ttyUSB0 | sudo tee -a /sys/module/kgdboc/parameters/kgdboc”, only to receive “No such device”. Supplementing ttyUSB0 for ttyS0 froze my computer solid and presumably passed all control to the serial port (which I don’t have the expansion/dock for). Result, kgdb only supports true serial devices.

I found that it was possible to write the ftrace results out over serial, however, all messages stopped when the USB->Serial and USB went to sleep. I tried to keep them awake by enabling CONFIG_USB_SUSPEND_OFF and hacking early returns into usb_serial_suspend and uart_suspend_port, but it didn’t seem to make any difference.

More debugging options were then added over successive builds, including; PM verbose, ACPI debug, ThinkPad ACPI debug, and Kobject debugging. The latter was a bad idea, as it swamped the console and logs making them unreadable. No further useful data was produced by enabling them.

Finally I decided to put an early return to the sdhci suspend routine, just in case the thread of execution was reaching it - this worked, but resume was not possible, therefore neither was access to the logs. An early return in the resume routine allowed a successful resume of the system, providing me with access to log functionality. Now I had printk I would follow the execution path with an advancing return statement, placing breadcrumbs (printks) behind it. This is where debugging became _slow_ but steady.

Due to the fact that sdhci functionality is plugged in as a module, debugging pointers to functions was a little tricky. In the end I compiled all MMC/SD code into the kernel as a built-in. This allowed me to look them up in the System.map. It then seemed as though the MMC remove routine was being called recursively. I placed in a static variable and an “if (count++ > 10) return;” to unwind any accidental recursion.

I followed the thread of execution though the following path:

(Remember: I found this out by advancing a return statement and commenting out functions as I went. This kind of debugging is very _slow_ progress.)

sdhci_suspend_host    - drivers/mmc/host/sdhci.c
mmc_suspend_host      - drivers/mmc/core/core.c
mmc_sd_remove         - drivers/mmc/core/sd.c
mmc_remove_card       - drivers/mmc/core/bus.c
device_del            - drivers/base/core.c
bus_remove_device     - drivers/base/bus.c
device_release_driver - drivers/base/dd.c
mmc_bus_remove        - drivers/mmc/core/bus.c
mmc_blk_remove        - mmc/card/block.c
del_gendisk           - fs/partitions/check.c
invalidate_partition  - block/genhd.c
fsync_bdev            - fs/block_dev.c
sync_filesystem       - fs/sync.c
sync_quota_sb         - fs/quota/quota.c
ext2_sync_fs          - fs/ext2/super.c
ext2_sync_super       - fs/ext2/super.c
mark_buffer_dirty     - fs/buffer.c
set_page_dirty        - fs/buffer.c
radix_tree_tag_set    - lib/radix-tree.c
root_tag_set          - lib/radix-tree.c

I eventually found myself in an inline function called root_tag_set, which contained the following line of code.

  root->gfp_mask |= (__force gfp_t)(1 « (tag + __GFP_BITS_SHIFT));

I tested this line to oblivion, trying to find out why it was failing. It turned out that it wasn’t failing at all, but it was making a difference. When this line was commented out suspend/resume would work perfectly, with the code left in, the system would come to a crashing halt. Another piece of code must be using the tag bit later down the line. The tag bit is used to mark memory pages which are dirty, or under write back [1]. Result, false alarm.

So I backed up to sync_filesystem and followed through another failing route I found. This went as follows:

sdhci_suspend_host        - drivers/mmc/host/sdhci.c
mmc_suspend_host          - drivers/mmc/core/core.c
mmc_sd_remove             - drivers/mmc/core/sd.c
mmc_remove_card           - drivers/mmc/core/bus.c
device_del                - drivers/base/core.c
bus_remove_device         - drivers/base/bus.c
device_release_driver     - drivers/base/dd.c
mmc_bus_remove            - drivers/mmc/core/bus.c
mmc_blk_remove            - mmc/card/block.c
del_gendisk               - fs/partitions/check.c
invalidate_partition      - block/genhd.c
fsync_bdev                - fs/block_dev.c
sync_filesystem           - fs/sync.c
sync_blockdev             - fs/block_dev.c
filemap_write_and_wait    - mm/filemap.c
filemap_fdata_write       - mm/filemap.c
filemap_fdata_write_range - mm/filemap.c
do_writepages             - mm/page-writeback.c
generic_writepages        - mm/page-writeback.c
write_cashe_pages         - mm/page-writeback.c
writepage                 - mm/page-writeback.c
blkdev_writepage          - fs/block_dev.c
block_write_full_page     - fs/buffer.c

I’m sure you can imagine the kind of debug vomit you’d expect from placing printks in here. Result, another dead end.

Unsure if it was happening during SD unload or something was referencing it after in the common code areas, I decided to insert debug in the tail-end of sdhci_suspend_host. But what? printks were no good. I had to run the code with the error left in, in which case I wouldn’t receive any output unless there was a successful resume. I ended up stealing a small for-loop from cking which flashed the CapsLk LED situated on the monitor several times. Result, the error was occurring somewhere during the SD unload.

I found another piece of information today which may prove to be useful:
  - This problem only occurs when _both_ the HDD and SD drives are mounted. If either one of them are unmounted, the problem does not surface. I uncovered this by booting a live CD and suspending. Under normal live CD operation the system suspends without issue. When I manually mount the HDD, then try to suspend the computer crumbles.

There are many applications in the GNOME desktop which we use and love too. Some of them are developed upstream at GNOME while some are developed by enthusiastic people downstream at usually distro level. Many of these downstream developers have closeness to one or more distro. Many of them are even made by individuals in their spare time and they are not started for ending up as official GNOME app.

Example:

Solang, GNote are developed mostly by people who have connections with Fedora.

Zeitgeist, Gnome Do are developed by individuals/group closer to Ubuntu.

Now read the rant from Seif Lotfy regarding Zeitgeist not being part of GNOME 3.

What we got from upstream GNOME

Nautilus, Evolution, NetworkManager, Epiphany

What we got from downstream/individual developers?

GNOME Do. Zeitgeist/GAJ, Docky, Pinta

Which apps do you love the most? From upstream or downstream ones? Fodder for thought.

For everyone who is not able to understand this problem, just have a look here:

When you select “Safely Remove Drive” the device gets unmounted and powered down. You need to have a look at the bug report at Gnome Bugzilla where the GNOME dev explains why they need to put all three options.

A furthur explanation is given as

First, some terminology:

"safely remove drive" is intended to mean "power down the drive so it is safe
for removal". Right now this is done by suspending the USB port but there's a
patch pending for 2.6.32 for powering down the port completely. The effect is
that the device powers off, specifically that leds on the device (if any) will
be powered off. Many people have asked for this feature, certain (rare) pieces
of hardware seems to require it.

(There might be a better term for option than "safely remove drive" but it's a
bit late to change it for 2.28.x. - and, for the record, it's similar in name
and functionality to what Windows is using.)

Second, "eject" means "eject the media" (for e.g. optical drives) or "make the
annoying 'not safe to remove' banner on the device go away" (for phones, ipods,
ebook readers etc.).

Third, "unmount" means well.. our beloved geek-word... also known as 'umount'.
Windows, I'm told, nowadays uses "Dismount". We could remove this geek-comfort,
it might be nice.

I can understand the problem, but keeping 3 options is nothing more than confusing the users. Even people who use Linux on day-to-day basis also think for a second when encountered with this option list.

We are here to make Ubuntu easier to use for the masses. This is surely a roadblock even though keeping all the three options might be necessary.

The simplicity and non-confusing nature of windows was the key reason of the growth if you step out of the anti-MS campaign. Non-techies never get confused with removing media on windows. Why is Ubuntu behind?

There is a papercut bug on Launchpad which asks whether the current way of handling archives too complicated for new users?

Let me express my experiences. I hardly double-click and open an archive since I know it would open the roller and then I would have to extract it, give the location and more loads of work. The way I work is right-click, “Extract Here”. It is a boon in disguise. Most users don’t want to use another archive manager as they are comfortable with the file explorer. Most of them want to view the contents in the explorer.

People resist change. Even Firefox awesome bar was critizied like hell, but the Firefox devs were didn’t budge. They added more finer control on the bar search and now no one complains. People get used to it.

What do you guys think? Is this a good idea?