Process of creating first patch

So this blog pretty much covers everything about the process and commands which I follow for the practical session of my Linux kernel first patch workshop.

I usually prefer to go for the simple syntax errors [suggested by checkpatch.pl] for the first patch practice. Also, usually it’s a good idea to send clean up patches for the files in staging directory only. Specifically when you are newbie and sending a patch in a kernel for the 1st time.

Note that this is just a list of commands, for more detailed versions and things I explained, you can refer to the slides and first patch tutorial. Also, make sure to run all of these commands under the source code directory of the Linux kernel and where Makefile resides. Do not run these commands under specific directories.

Step 1:

Run checkpatch.pl over the directory of your choice under the staging to edit a file.

perl scripts/checkpatch.pl -f drivers/staging/media/bcm2048/* | less

Step 2:

Check the code. Use following command to see who did that change and why. This is
useful in a practice to understand the approach of a code author.

git blame -L x,y file_path.c // x,y specifies the Line numbers

e.x  git blame -L 30,32 filr_path.c 

This will give you a list of commits. You can use ‘git show <commit id>’ to check the
whole commit.

Step 3:

Once done with changing the code, compile the file. It should produce the .o file without any errors.

make file_path.o

Sometimes you may want to run ‘make file_path/’. For example, you can run ‘make drivers/staging/media/bcm2048/’ .

Step 4:

If the file successfully compiles then go for creating a patch. For that, first
add your changes in a git.

Git add <file_path>

Step 5:

Now, create a commit.

Git commit -s -v

-s adds signed off by line and -v adds git diff in the commit you are creating.
You can read about the patch philosophy over here. For the patch best practices, refer to patch philosophy.

Step 6:

Create a patch now.

Git format-patch -o /tmp/ HEAD^

This will give you temporary .patch file. ex.

Step 7:

In the LKML, we prefer that new patches should not introduce any syntax warnings.
So, run checkpatch.pl over your temp .patch file. And solve sytax errors if there
are any.

scripts/checkpatch.pl <.patch file>

Step 8:

Get the list of maintainers and mailing lists to send that patch.

git show HEAD | perl scripts/get_maintainer.pl –separator , –nokeywords –nogit –nogit-fallback –norolestats –nol

Step 9:

Send the patch using git send-email.

git send-email –to= –cc= <.patch file>

To setup git send-email you can go for this blog post.

Also, few other commands which I use while giving first patch workshops are here:

To check the modified file:

git status and git diff

To check the commit after creating it.

git log

I’ll write separate posts explaining what to do when going for the v2 of the
patch.

Advertisements

Sending patches with git send-email

In this post I’m explaining how to send Linux kernel patches with git send-email using your gmail account. This is also useful for my Linux kernel workshops and generally for kernelnewbies as well. So, here we go:

First, you need to tell git what your name and email address is, so that it can be used in the authorship information in the git commit. Create a file called .gitconfig and add lines like these to it:

[user]
           name = Vaishali Thakkar
           email = vthakkar1994@gmail.com

Make sure that the email you specify here is the same email you used to set up sending mail.

The next step is to indicate to git-send-email which SMTP server it will use to send your patches and to specify its parameters e.g. encryption protocol, port etc. So, for gmail accounts add following lines in .gitconfig:

[sendemail]
           smtpuser = vthakkar1994@gmail.com
           smtpserver = smtp.googlemail.com
           smtpencryption = tls
           smtpserverport = 587

So, your final .gitconfig file should look something like this:

[user]
           name = Vaishali Thakkar
           email = vthakkar1994@gmail.com
[sendemail]
           smtpuser = vthakkar1994@gmail.com
           smtpserver = smtp.googlemail.com
           smtpencryption = tls
           smtpserverport = 587

There are many other options which you can add for the same. Those can be find here. But with above setting at least you should be able to send the patches. You can setup the password in the config file itself with ‘smtppass’ , if you don’t want git to send ask your password every time while sending patches.

After this, try sending a .patch file to yourself by using following command. If that doesn’t work then check if you have some typo in .gitconfig.

git send-email –to file.patch

Also, for the new ubuntu distros send-email might not work. This is because Linux distributions often like to split up packages to avoid dependencies needed only by optional functionality (like send-email). So, in that case you can install it with the following command in ubuntu:

sudo apt-get install git-email

Also, if you have If you have multi-factor authentication setup on your gmail account, you will need to generate an app-specific password for use with git send-email. Visit this site to setup an app-specific password. Usually allowing less secure apps works as well.

Once, git send-email is working properly, you can just go for sending patches to the list of maintainers and mailing lists after running scripts/get_maintainer.pl over a patch.

[Outreachy] Tips for the kernel newbies

Next round of Outreachy has started from September 12, 2016. I usually get lot of mails and messages through social networking sites regarding Outreachy and generally for the contribution to Linux Kernel. I worked as an Outreachy intern in Round 10 and last summer I also worked as a GSoC mentor under the Linux Foundation. GSoC project I mentored was also related to Linux Kernel. So, I guess it is probably right time to give few tips to prospective applicants of the next Outreachy round. Some of these tips also applies to every newbie who wants to contribute to Linux Kernel.

General tips:

  • Start early so that you have enough time to understand how community works in general and to decide which projects you are interested in.
  • Do not hesitate to ask questions. But keep in mind that mentors also expect you to do some sort of search before asking a question. So, it is important to understand  what questions to ask. The paper How To Ask Questions The Smart Way by Eric  Raymond and Rick Moen is an excellent guide.
  • Take mentors’/reviewers’/maintainers’ comments positively. And work on them to avoid repeating same mistakes again. Also, learn from others’ mistakes. I am still used to read comments of developers on others’ patches so that I don’t repeat the same mistake in my patch. I followed the same rule during my application round as well. So, read comments of mentors on other applicants’ patches and try to follow them as well.
  • If something is not clear in the comment then please ask. It is better to ask at that time than repeating the same mistake again. This will save your time.
  • Be nice and respect others’ point of views. Definitely we like to discuss on technical parts but arguing politely will help you to create bonding and it is necessary for the healthy community as well.
  • The culture in a particular open source community may vary from that in other communities. Kernel community is famous for their frank opinions. Generally the comments on your patch is for your code or the change you are proposing, and not for the person as an individual. So,never take those comments personally. Take them positively and try to improve the quality of code for the next contribution.
  • Engage in community. We like when applicants helps each other. Never look at this as a competition. In the end, we all are going to learn something. This is true for both mentors and applicants.

Kernel contribution or patch related tips:

  • First kernel patch tutorial is your friend. This includes everything from installing things to sending your patches properly. Read it very carefully.
  • Usually in Outreachy we advise applicants to start from checkpatch.pl issues just to give  them an idea about how community works and to help them get started with contributing to the Linux Kernel. But not every checkpatch.pl warning/error is right. So please try to understand why that warning is there instead of fixing them blindly. There are false positives as well.
  • Do not top post while responding to mails. Look at the details of how to respond to mails inline.
  • Do not copy commit log and subject. You can check the other patches for the reference but copy pasting will never help.
  • For the subject line always run ‘git log’ to check what kind of subject lines are used for the particular file.
  • Quantity of patches matters but quality of patches is important as well. Move from checkpatch.pl issues once you get the idea of how things are working. This will help you to learn more.
  • Usually in the Linux kernel projects, we have small tasks for the applicants by mentors. Work on them and ask questions to mentors if something is not clear.
  • Be consistent in sending patches. Being in touch with the mentors help them to understand your perspective for the project.
  • Two important things to do before sending every patch:
    1. Always compile-test your patch before sending it to the maintainers/mailing lists.
    2. Always run checkpatch.pl on your patch.

    Half of the time this will show you if your change is correct or not.

Application related tips:

  • Never lie about your commitments for the internship period. Be clear and discuss this with mentors of the project you are applying.
  • Mention about what kind of work you have done and how this will help you achieve your goal in the project. The point of mentioning this tip here is, you can have idea about what kind of patches you should send during the application period.

Last but not least, always remember that we all were newbie at one point of time. So, never feel embarrassed or bad when something is not working. You are amazing and we are thankful to you as you are helping us to improve our community or code or even ourselves as well.

Welcome to the kernel community and all the best everyone!

Yes, It’s my glass of juice!

Hello people!

I am back with some more interesting stories of my journey with the Linux kernel and open source. I have like dozen drafts saved in my google doc for the blog posts but none of them find their place to be published here for one or other reason.

Actually since I have relocated to the Bangalore, I am busy with the work on weekdays and in attending/speaking at events on weekends. It’s almost 6 months since I joined Oracle as a Linux kernel engineer and the journey so far has been fantastic. But I am not going to write about that in this blog [may be some other day!]. This time, I would like to share about my experience at speaking local/national events/conferences. So far, here is a list where I spoke at in last 5 months.

1. Took a session on ‘Contributing to Linux kernel’ at first Bangalore kernel meetup [slides, blog]
2. Conducted a workshop on ‘Contributing to Linux kernel‘  and took a session on ‘Getting started with contributing to open source‘ with Tapasweni Pathak at NIT, Calicut in FOSSMeet, 2016 [slides]
3. Took a session on ‘Getting started with contributing to open source’ at NIE, Mysore [slides]

With this I also had some casual webinars with some university students whenever it was not possible for me to present a talk in person. After all of this, I thought it’s good to write answers of 2 W’s. Why and What! So, let me write an answer of them.

1. Why I decided to speak at  events/conferences?

To get out of my comfort zone :

Yes, I thought to start speaking at conferences because I wanted to get out of my comfort zone. During my early college days, I was someone who could’t even raise a hand during lectures to ask the questions because I was used to think that I am supposed to know all of the answers. I think this feeling is common with many people. But Open source and Outreachy taught me that no question is a silly question and it’s good to speak up when you have valid points. Although that was all limited to mailing lists or IRC discussions. So, my primary motive was to to kill my fear of public speaking.

To share and improve my knowledge:

I think it’s a false assumption that when you are speaking at the event, you are just sharing your knowledge. Actually it’s more about improving your knowledge than sharing it. When you decide to give a public talk, you need to be ready for each and every question that can come in to mind of audience while attending your session. And sometimes you need to be prepare for out of context questions as well [Someone asked me about Iphone and Nexus 6P at one of my talk :D]. Actually this improves your knowledge about things and how you look at things. Also, the talks I have given till now had more percentage of students [2 university talks]. And they had lots of questions about which I didn’t even give a thought before that. So, this actually helped me to become more accurate about things and information while talking about them or including them in my talks.

– To be more socially active:

I am an introvert and I found it difficult to start conversation with strangers [I still do]. But I realized that speaking at public events/conferences can help to become more socially active. Because you are the one from whom people are expecting the answers of their questions and wants to discuss about your talk or things in general. On top of that, best part is you don’t need to worry about saying ‘Hi’ first. 🙂

– Giving back to the community:

During my open source journey, people have helped me a lot in many ways. And by giving public talks I wanted to give my few cents to an amazing open source community. And I am counting this as a success at some point because we have got contributions from the students/individuals in the Linux kernel after that.

2. What I learnt from speaking at these events/ conferences?

– It’s ok to be nervous:

Yes, it is pretty natural to be nervous but the important part is how you are going to handle that nervousness. One of the thought which helps me in handling this is ‘I have something to say and I am there because I want people to hear that’. And you know what, this actually helps me in communicating my ideas to the audience very clearly with the confidence.

– Became more confident by travelling alone:

I did my schooling and graduation from my hometown only. So, travelling alone never happened to me before relocating to Bangalore. But I was always sure that I can handle the things by my own [although my parents were not :D]. So, actually after this my parents have become more confident about myself than me.

– Learnt to talk with strangers

Either it’s about talking with the co-passengers or organizers of the events or attendees, I learnt how to talk with strangers. And that actually led me to some interesting conversations. For example, at NIE I was free after the talk, so I went to roam around the city with some girl students. And we actually had some nice discussions about how they can encourage more girls to participate in coding and which kind of sessions they can conduct at their universities. I got to know about different perspectives of the girls at university level.

So, yes the important thing I realized after speaking at these events is ‘It’s my glass of juice’ and if I can do this then anyone can I guess. Now I am hoping to speak at some international conferences in upcoming months [I was previously invited at LinuxCon Europe, 2015 and Devconf cz, 2016 but couldn’t make it due to some reasons.]. I am also open for giving talks at Indian universities as well. Because I believe students are the future developers and there are universities where students does not even have any idea about open source and how open source projects works. I have also seen cases where students have never used Linux during their 4 years of undergraduation [yes, not even once]. And I am hoping to change this scenario in India.

Finally, thanks to Outreachy and organizers for sponsoring the trips. And with this, I am signing off for now. Ping me if you have any questions about my slides or talks or open source/Linux kernel.

Thanks for passing by.

P.S. The only reason behind the title of this blog post is, I don’t like tea/coffee. 😀

 

Devm functions and their CORRECT usage

Hi,

Lets start this blog with some good news. Guess what, I won the Linux Foundation Training Scholarship under Kernel Guru category. I am so excited for taking Linux Foundation training class. They announced names of scholarship winners in LinuxCon NA which was in settle last week and it came out that I am a youngest scholar who won this scholarship. 🙂

So, lets come to the topic of this blog post. Over the last couple of weeks, I worked on devm functions and their incorrect usage. I couldn’t make it to update my blog properly regarding the same. But in this blog, I have tried to write good summary of my work on devm functions and by adding links wherever possible for someone who is more interested in the discussions and patches. First of all, lets start with introducing devm functions.

Devm Functions: What, Why and Where

There are some common memory resources used by drivers which are allocated using functions and stored as a linked list of arbitrarily sized memory areas called devres. Some of this memory is allocated in the probe function. But all of this memory should be freed on a failure patch or before the driver is detached. Else, the driver will leak resources and wastes main memory.

There are couple of reasons behind introducing devm functions. First is of course about resource leaking. It can be possible that if anything fails in the middle of probe function, it frees anything allocated. Also, remove function is just a duplicate code of probe’s error handling. And most important thing is it is hard to spot problems in failure handling with non-managed resource allocation. So, we need managed versions of such functions = devm functions. Devm functions basically allocates memory in a order resources are allocated and deallocates those memory automatically in a reverse order. Here, is a link of slides which can be useful to understand devm functions and their usages.

Problems I encountered or came across while looking in to devm functions

1. Basically sometimes what happens is developers want to use devm functions because they think that it is better to use managed resource functions but they don’t know how to use it and end up with messing things. For example, here is one of my patch. In this file, we already have devm_snd_soc_register_card and as I said devm functions automatically handles when to free memory. So, actually we don’t need to use function for unregistering the card. I have sent some patches for similar cases and I have provided links of accepted patches at the end of the blog.

2. While working on devm functions, I encountered that many files are using devm_free_irq in remove functions with the devm_request_irq in probe function. One can understand that we need devm_free_irq in remove function when we don’t have devm counterparts for each allocated resource function and it needs to occur before something else to be released. That can be a case where we need to call devm_free_irq explicitly. But not all cases are like that. Here is a link of one of such discussion on a case where use of devm_free_irq is not at all necessary. There are some other such interesting cases for devm_free_irq and but one need to look closely for each case. Also, it can be possible that there are some other such cases for other devm functions. One another example is this.

3. Sometimes it is possible that memory lives within a single function and frees after some line of code. Then we don’t really need devm functions because it just wastes extra memory used for devres structures and few extra cycles for maintaining them. Here, is such example.

Here, are those Coccinelle semantic patches which I used to detect such cases. File devm_entry_points is used to detect problematic usage of devm functions and devm_opportunity is used for finding files which are still using non-managed functions whose devm counterparts already exists.  I worked on each case encountered by devm_entry_points but not going to discuss it here. In case you have any questions please ask me here and I’ll be happy to answer them.

I sent many patches using both of these files and for some which came in to my way while working on devm functions. Here, is a links of some of my accepted patches for the reference and there are many more in their way to be accepted. All of my accepted patches can be found here.  I am going to send some more as still there are many opportunities where devm functions can be used.

I know understanding devm functions is little bit complicated at first but once you will understand them properly, you can enjoy the beauty of them. Please ping me for any queries regarding devm functions or my patches.

Links:

  1. power_supply: bq24735: Convert to using managed resources
  2. ASoC: tegra: Use devm_clk_get
  3. ASoC: tegra: Convert to managed resources
  4. ASoC: davinci-vcif: Use devm_snd_soc_register_component
  5. crypto: sahara – Use dmam_alloc_coherent
  6. ASoC: rockchip: i2s: Adjust devm usage
  7. ASoC: samsung: Remove redundant arndale_audio_remove
  8. ata: pata_arasam_cf: Use devm_clk_get

Update – 1: I have continued working on devm functions even after the internship. So, now there are many patches which are accepted in the mainline kernel. And still there are many opportunities where one can go for working on. So, here is a link of my all patches. One can go there and check the things for better understanding.

Macro builtin_platform_driver

This post is for introducing use of macro builtin_platform_driver. I just came across it while solving cases of module init/exit boilarplate code. Paul Gortmaker introduced this macro. One can see many patches with his name in Linux Kernel git tree. So, lets discuss the reason behind introducing it and where we can use it. I am not going in to that much detail, instead I am providing links so that one can go there and understand what is happening here.

Why we need macro like builtin_platform_driver?

Basically there are increasing number of non-modular drivers which are using module_driver type register functions. And there are several downsides of using this. One can see that commit from Paul Gortmaker here which has very infromative commit log.

Where we can use this macro?

One can go for checking in Kconfig file while handling any driver. And see if the driver is configured with a Kconfig option that is declared as a bool. If that is the case then one can go for using this macro for that file. But yes one need to make sure that if the driver depends on other drivers then all of them are declared as bool. Usually we have tristate for module drivers.

How I came across this macro?
When I was working on module init/exit boilarplate issue, I just saw some patches from Paul Gortmaker. Also, while handling one case Paul Bolle suggested me to use this macro. Although I knew that this macro exists, I was not sure about using for that particular file. And I ended up using it for file drivers/hwtracing/coresight/coresight-replicator.c. My patch can be found here.

Can Coccinelle help to handle such cases?
No, Coccinelle can’t help to detect such cases because for each file one need to check Kconfig options in Kconfig file. So one need to do it by hand. But yes Coccinelle can help you with transformation part. One can write semantic patch accordingly. I have added some semantic patches on which I worked in my github account. One can use them and change them according to their need.

Ether device API functions and some other hacks

Hi again

I worked on ether device API functions in a last month before module init/exit cases. Mainly I worked on 3 functions and got understanding of some others. I am going to talk about some other functions along with ether device API functions in this blog. So, this is going to be somehow managed blog.

1. eth_zero_addr:

This is a ether device API function which replaces memset to assign zero address. Linux kernel community prefers to using eth_zero_addr function instead of memset to assign the zero address to the given array. The function definition goes like this:

static inline void eth_zero_addr(u8 *addr)
{
memset(addr, 0x00, ETH_ALEN);
}

There were like 10-12 files having such cases. I sent patches for all of them. Most of them are now added in a kernel tree. Here, is a semantic patch which I used to do this change. It is preety simple. Only thing is sometimes value ‘6’ is used as ETH_ALEN. In that case one need to check that it is really a network code and in a memset 6 means ETH_ALEN.

@eth_zero_addr@
expression e;
@@

-memset(e,0x00,\(6\|ETH_ALEN\));
+eth_zero_addr(e);

2. eth_broadcast_addr:
This is also ether device API function and replaces memset. Only difference is this function is used to assign the broadcast address to the given address array. There were only 2-3 such cases. I handled all of them too. Semantic patch I used is as follows:

@eth_broadcast_addr@
identifier e;
@@

-memset(e,\(0xff\|0xFF\|255\),\(6\|ETH_ALEN\));
+eth_broadcast_addr(e);

3. eth_hw_addr_random:

This is an intersting function. It generates random Ethernet address [MAC] to be used by a net device and set addr_assign_type so that state can be read by sysfs and be used by userspace. Now, definition of this function goes like this:

static inline void eth_hw_addr_random(struct net_device *dev)
{
dev->addr_assign_type = NET_ADDR_RANDOM;
eth_random_addr(dev->dev_addr);
}

Basically I found this case from the deprecated function list. There were like 63 uses of random_eth_addr in linux 3.0 and now very less uses are there. Primary reason behind this was here random_eth_addr is changed to eth_random_addrfor the consistency purpose. So, there were many commits proposing that change. But when I looked in to detail then found a large bunch of commits from Danny Kukawka in 2012, replacing random_eth_addr with eth_hw_addr_random. Although in each case one need to check many things at a time because sometimes original call of random_eth_addr/eth_random_addr do not make assignment to NET_ADDR_RANDOM. And in some cases we have seen that merging the call and the assignment is not correct. So, from my side I can just compile test the change. And in each case I need to write about this possibility under —. For the experiment I just sent a patch explaining the situation and it turned out positive. After tesing it maintainer applied the patch. That patch can be found here. I am going to send some more patches in a future.

4. Deprecated macro DEFINE_PCI_DEVICE_TABLE

This macro is actually deprecated and this comment is made in a header file too. I just accidentally came across this while reading some other code. And thought to handle it. Basically point is we can use struct pci_device_id instead of this macro. When I searched for the remaining use of this macro in a linux-next, it turned out that most of them were handled by Julia’s one of other intern in a Paris. Only 3 uses were remaining. I sent patches for them. One is applied in a local branch and others are on their way. This was a simple case too. Semantic patch I used is as follows:

@@
identifier a;
declarer name DEFINE_PCI_DEVICE_TABLE;
initializer i;
@@
– DEFINE_PCI_DEVICE_TABLE(a)
+ const struct pci_device_id a[]
= i;

5. Unnecessary function snd_pcm_lib_preallocate_free_for_all()

I picked this from deprecated functions list of Julia. Here, there is no point of using this function snd_pcm_lib_preallocate_free_for_all() because ALSA core takes care that all preallocated memory is freed when the PCM itself is freed. Only one case was left. I just sent a patch for it yesterday and to my surprise it was applied in a very next minute by Mark. 🙂 One can look at the patch here.

By the way, I came across non-modular version of module init/exit macros. Recently many such patches are added in a kernel tree. It is interesting to understand the reasons behind introducing and using such macro. I guess one separate post would be fine for that. I will post that blog in a 2-3 days. Also, I am working on fixing some usage of devm functions from last 2 weeks. I’ll write a blog about it in a next week too.

Till then signing off. Have a happy weekend.