dev-guides

Kernel Developer Workflow

After hearing the frustrations of students in past years who have tried to do their assignments directly in the VM console, using a barebones text editor, we have created a guide for building out an improved developer workflow. We present instructions for how to set up git shortcuts and configure a robust Vim dev environment.

By the end of this guide, you will be able to:

For the remainder of this guide, all changes will be made in the class virtual machine.

Vim Setup

Non-vim users: If you have a cool emacs/atom/etc setup, feel free to send us a guide and we will share it on the course website.

The remainder of this guide will be dedicated to building out a robust vim developer environment, so if you are not planning to use vim for this class jump straight to Cscope

Installation

First install the vim-gtk, clang, cmake, curl, python and python-dev packages on your system package manager.

For example, on Debian:

# apt install vim-gtk clang cmake curl python python-dev

We install vim-gtk over the default vim package because the default version is not compiled with python support, which is needed for the YouCompleteMe plugin later.

The Philosophy of Vim

Before diving into setup, I need to assert a critical point about working in vim: don’t use vim tabs! Instead, use buffers and windows.

Many of you taking this class have come fresh out of AP, where you first learned to use vim. When I took OS I was in the same boat; all I had with me was Jae’s .vimrc, a few keyboard shortcuts, and many bad habits. Unfortunately, no class really teaches how to use vim, and as a result its easy to become glued to functional, but inefficient methods.

For instance, if you repeatedly type :tabn/:tabp to navigate files, or if you have 10 different terminal windows open, each ssh’d to the same remote directory with 10 instances of vim open on 10 different files (aka me in AP) … your workflow is probably suboptimal.

Thus, beyond installing plugins, the aim of this guide is to enforce efficient vim usage patterns that will save you tremendous amounts of time in the long run.

People who are most familiar with IDEs or sublime/atom expect to use tabs with vim because that is how these other editors work: the relationship is 1 tab per 1 file, and opening/closing a file requires opening/closing a tab. Vim however is not meant to be used this way–it was designed to work within a single terminal tab by utilizing multiple windows + multiple buffers.

Definitions:

Here is an approximate translation from actions in sublime –> actions in vim:

Here’s an example of what this should look like. Note that I never open a new tab. Instead, I have 3 buffers open (top line) and 1 or 2 windows open. To view a file, I simply navigate the window I want to the appropriate buffer.

Demo

Install Vim-Plug

Now let’s get to installing plugins.

vim-plug is a plugin manager for Vim. There are other plugin managers out there but we’ll be using vim-plug to install the remaining plugins, so do this first!

And add the following to your ~/.vimrc:

" automatically downloads vim-plug to your machine if not found.
if empty(glob('~/.vim/autoload/plug.vim'))
  silent !curl -fLo ~/.vim/autoload/plug.vim --create-dirs
    \ https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim
  autocmd VimEnter * PlugInstall --sync | source $MYVIMRC
endif

" Define plugins to install
call plug#begin('~/.vim/plugged')

" All of your Plugins must be added before the following line
call plug#end()

After adding some plugins, run

:PlugInstall

Now all plugins will be installed inside ~/.vim/plugged and they will be automatically added to your vim runtimepath.

Remapping Leader

Vim has the concept of a “leader” key, which is used to program personal keyboard shortcuts. By default, it is mapped to \, which is a little difficult to reach. I suggest mapping it to either <space> or comma , (I prefer comma). In your ~/.vimrc, write either

let mapleader ="\<Space>"

or

let mapleader = ","

We will be using leader keys through the rest of the guide. When you see something like nmap <leader>l :bnext<CR>, it means we are mapping the keyboard shortcut <leader> + l to the action :bnext<CR>.

Bufferline Display:

Airline is a package to display status information such as git branch and what buffer you are currently viewing. Add this plugin to your ~/.vimrc.

" Define plugins to install
call plug#begin('~/.vim/plugged')

Plug 'vim-airline/vim-airline'

" Optional
Plug 'vim-airline/vim-airline-themes'

" All of your Plugins must be added before the following line
call plug#end()

And in your ~/.vimrc also add

" Airline
let g:airline#extensions#tabline#enabled = 1 " Enable the list of buffers
let g:airline#extensions#tabline#fnamemod = ':t' " Show just the filename

When it’s all done, you should have a header that looks like:

buffer

To easily maneuver around these buffers, add this to your ~/.vimrc:

" -----------Buffer Management---------------
set hidden " Allow buffers to be hidden if you've modified a buffer

" Move to the next buffer
nmap <leader>l :bnext<CR>

" Move to the previous buffer
nmap <leader>h :bprevious<CR>

" Close the current buffer and move to the previous one
" This replicates the idea of closing a tab
nmap <leader>q :bp <BAR> bd #<CR>

" Show all open buffers and their status
nmap <leader>bl :ls<CR>

Now to navigate your window left/right between buffers, just press <leader> + h or <leader> + l. To close a buffer, press <leader> + q.

Window navigation

To open a new window, enter:

You can also use ctrl-w + s or + v to make a horizontal or vertical split respectively

To navigate between windows, use: ctrl-w + h|j|k|l to navigate left|down|up|right.

Alternatively, add this to your ~/.vimrc so that you can move between windows using arrow keys:

" Use arrow keys to navigate window splits
nnoremap <silent> <Right> :wincmd l <CR>
nnoremap <silent> <Left> :wincmd h <CR>
noremap <silent> <Up> :wincmd k <CR>
noremap <silent> <Down> :wincmd j <CR>

To close a window, press ctrl-w + c

ctrl-p and Nerdtree

The Linux kernel is a massive code base, so for easy navigation we’ll want to add a filename grepper (ctrl-p) and file tree (Nerdtree)

" Define plugins to install
call plug#begin('~/.vim/plugged')
...

" Browse the file system
Plug 'scrooloose/nerdtree'

" Ctrlp
Plug 'kien/ctrlp.vim'

...

" All of your Plugins must be added before the following line
call plug#end()

Now add the following to your ~/.vimrc:

" ctrl-p
let g:ctrlp_custom_ignore = {
  \ 'dir':  '\v[\/](\.(git|hg|svn)|\_site)$',
  \ 'file': '\v\.(exe|so|dll|class|png|jpg|jpeg)$',
\}

" Use the nearest .git|.svn|.hg|.bzr directory as the cwd
let g:ctrlp_working_path_mode = 'r'
nmap <leader>p :CtrlP<cr>  " enter file search mode

" Nerdtree
autocmd vimenter * NERDTree
autocmd StdinReadPre * let s:std_in=1
autocmd VimEnter * if argc() == 0 && !exists("s:std_in") | NERDTree | endif
autocmd bufenter * if (winnr("$") == 1 && exists("b:NERDTree") && b:NERDTree.isTabTree()) | q | endif
map <C-n> :NERDTreeToggle<CR>  " open and close file tree
nmap <leader>n :NERDTreeFind<CR>  " open current buffer in file tree

Now if you want to search for and open a file, press <leader> + p. This is analogous to cmd+r in Sublime.

Nerdtree will show a file tree in the window on the left of your screen. To collapse/expand it, press ctrl + n. To expose your current working file in the file tree, press <leader> + n.

Altogether it looks like this:

File Navigation

YCM

Next we are going to add semantic auto-complete through a plugin called YouCompleteMe. This part is a little involved, so here’s a demo first so you can decide if it’s worth the setup:

YCM

Finally, install YCM

" Define plugins to install
call plug#begin('~/.vim/plugged')
...

" YCM
Plug 'Valloric/YouCompleteMe'

...

" All of your Plugins must be added before the following line
call plug#end()

Then,

$ cd ~/.vim/plugged/YouCompleteMe
$ git submodule update --init --recursive
$ ./install.py --clang-completer --system-libclang

And add the following to your ~/.vimrc:

" Modify below if you want less invasive autocomplete
let g:ycm_semantic_triggers =  {
  \   'c' : ['->', '.'],
  \   'objc' : ['->', '.'],
  \   'cpp,objcpp' : ['->', '.', '::'],
  \   'perl' : ['->'],
  \ }

let g:ycm_complete_in_comments_and_strings=1
let g:ycm_key_list_select_completion=['<C-n>', '<Down>']
let g:ycm_key_list_previous_completion=['<C-p>', '<Up>']
let g:ycm_autoclose_preview_window_after_completion = 1

let g:ycm_global_ycm_extra_conf = '<path/to/your/ycm_extra_conf'

set completeopt-=preview

If you want to see how this works, check out YCM-Generator and the original YCM github page. We are also providing an example ycm_extra_conf.py at YCM_EXTRA_CONF

Important Note: for YCM in the kernel to work properly you need to be inside the kernel directory when you activate vim. e.g.

$ cd <path-to-homework-assignment>/kernel && vim

Cscope

This is the final step: Function GOTOs and efficiently browing code.

Cscope is a code browser that works in your terminal and within vim. It is far more powerful than a standard grepper (such as the one at elixir). For example, Cscope can answer:

To install on Debian:

# apt install cscope

Verify installation succeeded by running cscope. This should open up the Cscope browser in your terminal window. To exit, use ctrl-d.


Now, To use cscope you will need to first build the cscope db, which can be done via:

$ make cscope

This will generate kernel/cscope.files, which lists all kernel files Cscope will include in its project database. Then

$ cd kernel/
$ cscope -b -q -k

This builds the cscope db, which is comprised of three files: cscope.files, cscope.in.out, cscope.out. Do not touch these!

To use the Cscope browser:

$ cd <PATH-TO-HW-REPO>/kernel
$ cscope -d

Use <Tab> to alternate between the menu and the list of matching lines. See manpage for more usage instructions of the Cscope browser.

Demo:

Cscope

To enable vim support, we need to add a new cscope_maps.vim file. First, if it doesn’t already exist, run

$ mkdir -p ~/.vim/plugged/

All .vim files in this directory will automatically be sourced into your ~/.vimrc. Then run:

$ cd ~/.vim/plugged && wget http://cscope.sourceforge.net/cscope_maps.vim

See this tutorial for more usage details. For now, I will tell you the single most useful feature we have just added: Function GOTOs! Now, when your cursor is over a function call, enter ctrl + ] and you will automatically jump to the function declaration in a new buffer. And to move your cursor back and forth, use ctrl + o (backwards) and ctrl + i (forwards). See demo video below:

Function GOTO Demo

My last recommendation is to use 2 terminal windows when working: one dedicated to vim/writing code, and the other dedicated to cscope/browsing code.

Fin

And that completes this developer workflow guide. Congrats if you’ve made it this far; hopefully the time spent reading this will be vastly outweighed by time you’ll have saved in this class and beyond.

You can also take a look at Howon’s example ~/.vimrc here.