LXR is one of powerful source code browsing tool. I am using it in my daily work.
To feel how it is like you can go to http://lxr.linux.no/linux/, Setting up a LXR in your PC or a Local server could speed up your search request.
I searched the net for instruction on how, the documentation was not that much good, and also there were few web links on how to.
I was using Ubuntu 10.04, so the steps should be straightly applicable to Ubuntu based systems. you can also used the same steps for non ubuntu distros, the configuration will be same for all. only difference will be in the dependency package installation. your distro may use different name for same package
Part 1: Dependency package installation
Install the postgresql database and it’s client apps:
sudo apt-get install postgresql-8.4 postgresql-client-8.4
Install the Xapian search engine library and its Perl bindings:
sudo apt-get install libxapian15 libsearch-xapian-perl
Install apache and its perl module and as mentioned in documentation, install some of the HTML/Web Perl modules:
sudo apt-get install apache2 libapache2-mod-perl2
Install two more CPAN modules needed by the indexing process. One of them is used to check the memory usage and the second to draw a shiny toolbar.
sudo apt-get install libcgi-simple-perl libcgi-ajax-perl libhtml-parser-perl libtemplate-perl sudo apt-get install libterm-progressbar-perl libdevel-size-perl
sudo apt-get install exuberant-ctags
Install Perl abstraction layer to a database and Perl DBI driver for the PostgreSQL database server
You will need git to get the lxrng, If you don’t have git in your system then follow the following command. If you have already then you can skip the next command
$ sudo apt-get install git-core
Part 2: Data base setup
Now my user name is ‘thalib’ that I am actually logged in with and the one that I did all the above commands with,
run following to create a database for lxr
thalib@mohamed:/home/web/lxrng$ sudo -i root@mohamed:~# root@mohamed:~# su - postgres postgres@mohamed:~$ createuser thalib Shall the new role be a superuser? (y/n) y postgres@mohamed:~$ createdb lxrng postgres@mohamed:~$ createuser www-data Shall the new role be a superuser? (y/n) n Shall the new role be allowed to create databases? (y/n) y Shall the new role be allowed to create more new roles? (y/n) n postgres@mohamed:~$ logout root@mohamed:~# logout
Part 3: Configuring LXRng.
Now its time for configuring lxrng.
Create a directory where you want to install LXRng. I install it in my /home/web dir, where I have plenty of space.
thalib@mohamed:/home/thalib$ cd /home thalib@mohamed:/home$ mkdir web thalib@mohamed:/home$ cd web
Clone lxr’s developer git tree as follows:
thalib@mohamed:/home/web$ git clone git://lxr.linux.no/git/lxrng.git
Now you should have the lxrng directory created in PWD. you can check it by “ls” command
thalib@mohamed:/home/web$ cd lxrng
Now you should install the following modules.
LXR comes with a template called lxrng.conf-dist. we can copy it to create lxrng config file named lxrng.conf
thalib@mohamed:/home/web/lxrng$ cp lxrng.conf-dist lxrng.conf
Below is the content of modified file for linux repository.
# -*- mode: perl -*-
# Lxrng Configuration file for linux repo
#
#
use LXRng::Index::PgBatch;
use LXRng::Repo::Git;
use LXRng::Search::Xapian;
my $gitrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
my $index = LXRng::Index::PgBatch->new(db_spec => 'dbname=lxrng;port=5432',
db_user => "", db_pass => "",
# table_prefix => 'lxr'
);
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/text-db/linux-2.6');
return {
'linux' => {
'repository' => $gitrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.1.2/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
'ver_list' => [$gitrepo->allversions],
'ver_default' => 'v2.6.33',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
};
Now we have create some directories for lxrng, this depends lxrng.conf file.
create cache directory text-db/linux-2.6 directory as we gave in the lxrng.con file the
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/text-db/linux-2.6');
'cache' => '/home/web/lxrng/cache',
thalib@mohamed:/home/web/lxrng$ mkdir text-db/linux-2.6 -p
thalib@mohamed:/home/web/lxrng$ mkdir cache
thalib@mohamed:/home/web/lxrng$ chmod 777 -R cache
[/sourcecode]
Note: make sure www-data user is able to read and write to text-db, cache dirs.
As we mentioned in the lxrng.conf, we have to create a repos directory and clone linux git as below
my $gitrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
Now create reops direcotry and clone the linux git repository
thalib@mohamed:/home/web/lxrng$ mkdir repos thalib@mohamed:/home/web/lxrng$ cd repos thalib@mohamed:/home/web/lxrng/repos$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git thalib@mohamed:/home/web/lxrng/repos$ ls 'ver_default' => 'v2.6.33',
to set the above variable in lxrng.conf use the below command and get the list of tags
thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ git tag
For me I am using the latest tag v2.6.33, which is latest and stable at the time I wrote this blog
thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ cd ../../ thalib@mohamed:/home/web/lxrng$./lxr-db-admin linux --init NOTICE: CREATE TABLE will create implicit sequence "charsets_id_seq" for serial column "charsets.id" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "charsets_pkey" for table "charsets" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "trees_pkey" for table "trees" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "releases_pkey" for table "releases" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "files_pkey" for table "files" NOTICE: CREATE TABLE / UNIQUE will create implicit index "files_path_key" for table "files" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "revisions_pkey" for table "revisions" NOTICE: CREATE TABLE / UNIQUE will create implicit index "revisions_id_file_key" for table "revisions" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "filestatus_pkey" for table "filestatus" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "hashed_documents_pkey" for table "hashed_documents" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "filereleases_pkey" for table "filereleases" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "symbols_pkey" for table "symbols" NOTICE: CREATE TABLE / UNIQUE will create implicit index "symbols_name_key" for table "symbols" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "identifiers_pkey" for table "identifiers" thalib@mohamed:/home/web/lxrng$ ./lxr-genxref linux
the above name linux in both lxr-db-admin and lxr-genxref command should match what you have given in the lxrng.conf file as shown below.
return {
'linux' => {
'repository' => $gitrepo,
Have a break and hang out with your friend’s for a while if you have a huge source repository like the Linux kernel.
I mostly do this operation during night. I used to start it before going to bed, by the morning it will be complete
thalib@mohamed:/home/web/lxrng$ cp apache2-site.conf-dist-mod_perl apache2-site.conf thalib@mohamed:/home/web/lxrng$ ln -s $PWD/apache2-site.conf /etc/apache2/sites-enabled/010-lxrng thalib@mohamed:/home/web/lxrng$ ls /etc/apache2/sites-enabled/ lrwxrwxrwx 1 root root 26 2010-05-09 08:09 000-default -> ../sites-available/default lrwxrwxrwx 1 root root 33 2010-05-09 08:40 010-lxrng -> /home/web/lxrng/apache2-site.conf
Then I have to customize it. Below is my customize apache2-site.conf file.
PerlInitHandler Apache2::Reload PerlOptions +GlobalRequest PerlRequire /home/web/lxrng/lxrng_mod_perl.pl PerlModule LXRng::ModPerl Alias /lxr "/home/web/lxrng/webroot/" <Directory "/home/web/lxrng/webroot/"> Options ExecCGI AllowOverride None Order deny,allow Allow from all <Files *> SetHandler perl-script PerlResponseHandler LXRng::ModPerl </Files> <Files favicon.ico> SetHandler send-as-is </Files> <Files robots.txt> SetHandler send-as-is </Files> </Directory> <Directory "/home/web/lxrng/webroot/.static/"> <Files *> SetHandler send-as-is </Files> </Directory>
Restart the apache server for the settings to get updated.
thalib@mohamed:/home/web/lxrng$ sudo /etc/init.d/apache2 restart
Now open your favorite browser and enter http://localhost/lxr or <url>/lxr or <IP Address>/lxr
That’s it, Done. You did it.
Part 4: Add more than one repository to LXRng.
For me as a embedded developer I need some more source repositories to be added. I will show no how to add more repos to the same page.
All you have to do is modify the lxrng.conf to add more git repositories.
# -*- mode: perl -*-
# Configuration file
#
#
use LXRng::Index::PgBatch;
use LXRng::Repo::Git;
use LXRng::Search::Xapian;
my $index = LXRng::Index::PgBatch->new(db_spec => 'dbname=lxrng;port=5432',
db_user => "", db_pass => "",
# table_prefix => 'lxr'
);
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/textdb');
my $linuxrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
my $ubootrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/u-boot/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
return {
'linux' => {
'repository' => $linuxrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.6.222/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache/linux',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
#'ver_list' => [$linuxrepo->allversions],
'ver_list' => ['v2.6.35'],
'ver_default' => 'v2.6.35',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
'uboot' => {
'repository' => $ubootrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.6.222/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache/u-boot',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
#'ver_list' => [$linuxrepo->allversions],
'ver_list' => ['v2010.06', 'v2010.06-rc3'],
'ver_default' => 'v2010.06',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
};
then create the dirs as follows
thalib@mohamed:/home/web/lxrng$ mkdir -p cache/linux thalib@mohamed:/home/web/lxrng$ mkdir -p cache/uboot thalib@mohamed:/home/web/lxrng$ mkdir -p text-db thalib@mohamed:/home/web/lxrng$ cd repos/ thalib@mohamed:/home/web/lxrng$ git clone <linux git reposioty> thalib@mohamed:/home/web/lxrng$ git clone <uboot git reposioty>
Now to create a database and generate it. since we are using single database for both repos creating only repos is enough so we run lxr-db-admin for linux or uboot alone which will creat the database
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin linux --init thalib@mohamed:/home/web/lxrng$ ./lxr-genxref linux thalib@mohamed:/home/web/lxrng$ ./lxr-genxref uboot
If you want to rebuild the db then you can use –reinit to reinitilizie it or –drop to delete it and then create using –init as show below
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --reinit
or
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --drop thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --init
If you are running the setting up lxr for Linux source pulled from git, doing lxr-genref will take huge time. in my AMD machine I started lxr-genref for all version, it was running all the night but not even completed the half of the sorce, and also this will take considerable amount of memory and will also be slow in when you use lxr.
In this case If you need only one version then you say to lxr through lxrng.conf file to generate database for particular versions only.
'ver_list' => ['v2.6.32', 'v2.6.33'],
I the above I have set the lxrng.conf to generate database for only 2.6.32 and 2.6.33
Hope this is useful. Now you can keep on adding new repos to lxrng.
Part 5: Updating lxrng database when new version of linux arives.
Case 1:
If you have set ver_list variable in lxrng.conf to allversion as below
'ver_list' => [$gitrepo->allversions],
then just goto the corresponding repository and do a git pull and come to lxrng directory and run following command, which indexes on the new modifications only.
./lxr-genxref uboot
Case 2:
If you have set ver_list variable in lxrng.conf to fixed like below
'ver_list' => ['v2.6.32', 'v2.6.33'],
then just goto the corresponding repository and do a git pull and get the latest tag using git tag command, updated the ver_list variable with the new tag you like to index eg. below
'ver_list' => ['v2.6.32', 'v2.6.33', 'v2.6.33'],
Then come to lxrng directory and run following command, which indexes on the new modifications only.
./lxr-genxref uboot
Hope this is helpful. Insha Allah.
Reference
http://lxr.linux.no/
http://darwish-07.blogspot.com/2008/02/howto-lxrng-on-ubuntu-710.html
http://lxr.linux.no/.static/contrib/lxr-notes-ubuntu.txt
LXR is one of powerful source code browsing tool. I am using it in my daily work.
To feel how it is like you can go to http://lxr.linux.no/linux/, Setting up a LXR in your PC or a Local server could speed up your search request.
I searched the net for instruction on how, the documentation was not that much good, and also there were few web links on how to.
I was using Ubuntu 10.04, so the steps should be straightly applicable to Ubuntu based systems. you can also used the same steps for non ubuntu distros, the configuration will be same for all. only difference will be in the dependency package installation. your distro may use different name for same package
Part 1: Dependency package installation
Install the postgresql database and it’s client apps:
sudo apt-get install postgresql-8.4 postgresql-client-8.4
Install the Xapian search engine library and its Perl bindings:
sudo apt-get install libxapian15 libsearch-xapian-perl
Install apache and its perl module and as mentioned in documentation, install some of the HTML/Web Perl modules:
sudo apt-get install apache2 libapache2-mod-perl2
Install two more CPAN modules needed by the indexing process. One of them is used to check the memory usage and the second to draw a shiny toolbar.
sudo apt-get install libcgi-simple-perl libcgi-ajax-perl libhtml-parser-perl libtemplate-perl sudo apt-get install libterm-progressbar-perl libdevel-size-perl
sudo apt-get install exuberant-ctags
Install Perl abstraction layer to a database and Perl DBI driver for the PostgreSQL database server
You will need git to get the lxrng, If you don’t have git in your system then follow the following command. If you have already then you can skip the next command
$ sudo apt-get install git-core
Part 2: Data base setup
Now my user name is ‘thalib’ that I am actually logged in with and the one that I did all the above commands with,
run following to create a database for lxr
thalib@mohamed:/home/web/lxrng$ sudo -i root@mohamed:~# root@mohamed:~# su - postgres postgres@mohamed:~$ createuser thalib Shall the new role be a superuser? (y/n) y postgres@mohamed:~$ createdb lxrng postgres@mohamed:~$ createuser www-data Shall the new role be a superuser? (y/n) n Shall the new role be allowed to create databases? (y/n) y Shall the new role be allowed to create more new roles? (y/n) n postgres@mohamed:~$ logout root@mohamed:~# logout
Part 3: Configuring LXRng.
Now its time for configuring lxrng.
Create a directory where you want to install LXRng. I install it in my /home/web dir, where I have plenty of space.
thalib@mohamed:/home/thalib$ cd /home thalib@mohamed:/home$ mkdir web thalib@mohamed:/home$ cd web
Clone lxr’s developer git tree as follows:
thalib@mohamed:/home/web$ git clone git://lxr.linux.no/git/lxrng.git
Now you should have the lxrng directory created in PWD. you can check it by “ls” command
thalib@mohamed:/home/web$ cd lxrng
Now you should install the following modules.
LXR comes with a template called lxrng.conf-dist. we can copy it to create lxrng config file named lxrng.conf
thalib@mohamed:/home/web/lxrng$ cp lxrng.conf-dist lxrng.conf
Below is the content of modified file for linux repository.
# -*- mode: perl -*-
# Lxrng Configuration file for linux repo
#
#
use LXRng::Index::PgBatch;
use LXRng::Repo::Git;
use LXRng::Search::Xapian;
my $gitrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
my $index = LXRng::Index::PgBatch->new(db_spec => 'dbname=lxrng;port=5432',
db_user => "", db_pass => "",
# table_prefix => 'lxr'
);
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/text-db/linux-2.6');
return {
'linux' => {
'repository' => $gitrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.1.2/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
'ver_list' => [$gitrepo->allversions],
'ver_default' => 'v2.6.33',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
};
Now we have create some directories for lxrng, this depends lxrng.conf file.
create cache directory text-db/linux-2.6 directory as we gave in the lxrng.con file the
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/text-db/linux-2.6');
'cache' => '/home/web/lxrng/cache',
thalib@mohamed:/home/web/lxrng$ mkdir text-db/linux-2.6 -p
thalib@mohamed:/home/web/lxrng$ mkdir cache
thalib@mohamed:/home/web/lxrng$ chmod 777 -R cache
[/sourcecode]
Note: make sure www-data user is able to read and write to text-db, cache dirs.
As we mentioned in the lxrng.conf, we have to create a repos directory and clone linux git as below
my $gitrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
Now create reops direcotry and clone the linux git repository
thalib@mohamed:/home/web/lxrng$ mkdir repos thalib@mohamed:/home/web/lxrng$ cd repos thalib@mohamed:/home/web/lxrng/repos$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git thalib@mohamed:/home/web/lxrng/repos$ ls 'ver_default' => 'v2.6.33',
to set the above variable in lxrng.conf use the below command and get the list of tags
thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ git tag
For me I am using the latest tag v2.6.33, which is latest and stable at the time I wrote this blog
thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ cd ../../ ./lxr-db-admin linux --init NOTICE: CREATE TABLE will create implicit sequence "charsets_id_seq" for serial column "charsets.id" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "charsets_pkey" for table "charsets" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "trees_pkey" for table "trees" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "releases_pkey" for table "releases" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "files_pkey" for table "files" NOTICE: CREATE TABLE / UNIQUE will create implicit index "files_path_key" for table "files" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "revisions_pkey" for table "revisions" NOTICE: CREATE TABLE / UNIQUE will create implicit index "revisions_id_file_key" for table "revisions" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "filestatus_pkey" for table "filestatus" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "hashed_documents_pkey" for table "hashed_documents" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "filereleases_pkey" for table "filereleases" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "symbols_pkey" for table "symbols" NOTICE: CREATE TABLE / UNIQUE will create implicit index "symbols_name_key" for table "symbols" NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "identifiers_pkey" for table "identifiers" thalib@mohamed:/home/web/lxrng$ ./lxr-genxref linux
the above name linux in both lxr-db-admin and lxr-genxref command should match what you have given in the lxrng.conf file as shown below.
return {
'linux' => {
'repository' => $gitrepo,
Have a break and hang out with your friend’s for a while if you have a huge source repository like the Linux kernel.
I mostly do this operation during night. I used to start it before going to bed, by the morning it will be complete
thalib@mohamed:/home/web/lxrng$ cp apache2-site.conf-dist-mod_perl apache2-site.conf thalib@mohamed:/home/web/lxrng$ ln -s $PWD/apache2-site.conf /etc/apache2/sites-enabled/010-lxrng thalib@mohamed:/home/web/lxrng$ ls /etc/apache2/sites-enabled/ lrwxrwxrwx 1 root root 26 2010-05-09 08:09 000-default -> ../sites-available/default lrwxrwxrwx 1 root root 33 2010-05-09 08:40 010-lxrng -> /home/web/lxrng/apache2-site.conf
Then I have to customize it. Below is my customize apache2-site.conf file.
PerlInitHandler Apache2::Reload
PerlOptions +GlobalRequest
PerlRequire /home/web/lxrng/lxrng_mod_perl.pl
PerlModule LXRng::ModPerl
Alias /lxr "/home/web/lxrng/webroot/"
<Directory "/home/web/lxrng/webroot/">
Options ExecCGI
AllowOverride None
Order deny,allow
Allow from all
<Files *>
SetHandler perl-script
PerlResponseHandler LXRng::ModPerl
</Files>
<Files favicon.ico>
SetHandler send-as-is
</Files>
<Files robots.txt>
SetHandler send-as-is
</Files>
</Directory>
<Directory "/home/web/lxrng/webroot/.static/">
<Files *>
SetHandler send-as-is
</Files>
</Directory>
Restart the apache server for the settings to get updated.
thalib@mohamed:/home/web/lxrng$ sudo /etc/init.d/apache2 restart
Now open your favorite browser and enter http://localhost/lxr or <url>/lxr or <IP Address>/lxr
That’s it, Done. You did it.
Part 4: Add more than one repository to LXRng.
For me as a embedded developer I need some more source repositories to be added. I will show no how to add more repos to the same page.
All you have to do is modify the lxrng.conf to add more git repositories.
# -*- mode: perl -*-
# Configuration file
#
#
use LXRng::Index::PgBatch;
use LXRng::Repo::Git;
use LXRng::Search::Xapian;
my $index = LXRng::Index::PgBatch->new(db_spec => 'dbname=lxrng;port=5432',
db_user => "", db_pass => "",
# table_prefix => 'lxr'
);
my $search = LXRng::Search::Xapian->new('/home/web/lxrng/textdb');
my $linuxrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/linux-2.6/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
my $ubootrepo = LXRng::Repo::Git
->new('/home/web/lxrng/repos/u-boot/.git',
release_re => qr/^v[^-]*$/,
author_timestamp => 0);
return {
'linux' => {
'repository' => $linuxrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.6.222/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache/linux',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
#'ver_list' => [$linuxrepo->allversions],
'ver_list' => ['v2.6.35'],
'ver_default' => 'v2.6.35',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
'uboot' => {
'repository' => $ubootrepo,
'index' => $index,
'search' => $search,
'base_url' => 'http://192.168.6.222/lxr',
# Must be writable by httpd user:
'cache' => '/home/web/lxrng/cache/u-boot',
'fs_charset' => 'iso-8859-1',
# Tried successively
'content_charset' => ['utf-8', 'iso-8859-1'],
'languages' => ['C', 'GnuAsm', 'Kconfig'],
#'ctags_flags' => ["-I\@$LXRng::ROOT/lxr-ctags-quirks"],
#'ver_list' => [$linuxrepo->allversions],
'ver_list' => ['v2010.06', 'v2010.06-rc3'],
'ver_default' => 'v2010.06',
'include_maps' =>
[
[qr|^arch/(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^include/asm-(.*?)/|, qr|^asm/(.*)|,
sub { "include/asm-$_[0]/$_[1]" }],
[qr|^|, qr|^asm/(.*)|,
sub { map { "include/asm-$_/$_[0]" }
qw(i386 alpha arm ia64 m68k mips mips64),
qw(ppc s390 sh sparc sparc64 x86_64) }],
[qr|^|, qr|(.*)|,
sub { "include/$_[0]" }],
],
},
};
then create the dirs as follows
thalib@mohamed:/home/web/lxrng$ mkdir -p cache/linux thalib@mohamed:/home/web/lxrng$ mkdir -p cache/uboot thalib@mohamed:/home/web/lxrng$ mkdir -p text-db thalib@mohamed:/home/web/lxrng$ cd repos/ thalib@mohamed:/home/web/lxrng$ git clone <linux git reposioty> thalib@mohamed:/home/web/lxrng$ git clone <uboot git reposioty>
Now to create a database and generate it. since we are using single database for both repos creating only repos is enough so we run lxr-db-admin for linux or uboot alone which will creat the database
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin linux --init thalib@mohamed:/home/web/lxrng$ ./lxr-genxref linux thalib@mohamed:/home/web/lxrng$ ./lxr-genxref uboot
If you want to rebuild the db then you can use –reinit to reinitilizie it or –drop to delete it and then create using –init as show below
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --reinit
or
thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --drop thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin uboot --init
If you are running the setting up lxr for Linux source pulled from git, doing lxr-genref will take huge time. in my AMD machine I started lxr-genref for all version, it was running all the night but not even completed the half of the sorce, and also this will take considerable amount of memory and will also be slow in when you use lxr.
In this case If you need only one version then you say to lxr through lxrng.conf file to generate database for particular versions only.
'ver_list' => ['v2.6.32', 'v2.6.33'],
I the above I have set the lxrng.conf to generate database for only 2.6.32 and 2.6.33
Hope this is useful. Now you can keep on adding new repos to lxrng.
Part 5: Updating lxrng database when new version of linux arives.
case 1:
If you have set ver_list variable in lxrng.conf to allversion as below
'ver_list' => [$gitrepo->allversions],
then just goto the corresponding repositry and do a git pull and come to lxrng directory and run following command, which indexces on the new modifications only.
./lxr-genxref uboot
case 2:
If you have set ver_list variable in lxrng.conf to fixed like below
'ver_list' => ['v2.6.32', 'v2.6.33'],
then just goto the corresponding repositry and do a git pull and get the latest tag using git tag command, updated the ver_list variabe with the new tag you like to index eg. below
'ver_list' => ['v2.6.32', 'v2.6.33', 'v2.6.33'],
Then come to lxrng directory and run following command, which indexces on the new modifications only.
./lxr-genxref uboot
Hope this is helpfull. Insha Allah.
Reference
http://lxr.linux.no/
http://darwish-07.blogspot.com/2008/02/howto-lxrng-on-ubuntu-710.html
http://lxr.linux.no/.static/contrib/lxr-notes-ubuntu.txt
April 30, 2012 at 7:12 PM
Hi Md Thalif,
I get the below errors after I did “./lxr-db-admin linux –init” in the step-3
————————————————
Can’t locate DBI.pm in @INC (@INC contains: /home/dennyg/lxrDenn/lxrng/lib /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at /home/dennyg/lxrDenn/lxrng/lib/LXRng/Index/PgBatch.pm line 27, line 55.
BEGIN failed–compilation aborted at /home/dennyg/lxrDenn/lxrng/lib/LXRng/Index/PgBatch.pm line 27, line 55.
Compilation failed in require at configuration file line 6, line 55.
BEGIN failed–compilation aborted at configuration file line 6, line 55.
————————————————
Any idea as what this could be?
Thanks,
Denny G
May 11, 2012 at 10:58 AM
From the error messsage
It looks like DBI perl module is missing. Install it first to continue.
April 30, 2012 at 7:06 PM
Hi Thalif,
I followed instructions upto -
1 thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ git tag
01 thalib@mohamed:/home/web/lxrng/repos/linux-2.6$ cd ../../
03 thalib@mohamed:/home/web/lxrng$./lxr-db-admin linux –init
and when I give the command ”./lxr-db-admin linux –init” in part 3 , I get the following errors
———————-
Can’t locate DBI.pm in @INC (@INC contains: /home/dennyg/lxrDenn/lxrng/lib /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at /home/dennyg/lxrDenn/lxrng/lib/LXRng/Index/PgBatch.pm line 27, line 55.
BEGIN failed–compilation aborted at /home/dennyg/lxrDenn/lxrng/lib/LXRng/Index/PgBatch.pm line 27, line 55.
Compilation failed in require at configuration file line 6, line 55.
BEGIN failed–compilation aborted at configuration file line 6, line 55.
———————-
Regards,
Denny
October 3, 2011 at 12:37 PM
[...] How to setup LXR – Step by Step guide « Mohamed Thalib's Blog [...]
January 3, 2011 at 11:35 AM
[...] How to setup LXR – Step by Step guide July 2010 8 comments and 1 Like on WordPress.com, [...]
September 15, 2010 at 10:20 PM
Thank you very much my friend, you are very kind in sharing this useful information with? others…. The details were such a blessing, thanks.
September 13, 2010 at 3:29 AM
This is a great post and may be one that should be followed up to see what are the results
A neighbor sent this link the other day and I’m excitedly waiting your next piece of writing. Carry on on the first-class work.
August 31, 2010 at 1:42 AM
Sorry for another post, but the parser killed my comments in my section 2. that’ll teach me to use html tag identifiers as comment delimiters. Changed them to a sharp below:
mkdir /home/web/lxrng/repos/random-drv
cd $_
git init
# copy first version of driver code into /home/web/lxrng/repos/random-drv)
git add .
git commit -a -m “random-linux-driver 1.0″
git log
# note the number of the last commit (at the top if there’s a lot) )
git tag v1.0
# if you have more versions to do delete the files in the dir, but leave the .git folder,
# then go back through the loop again and copy the next version into the folder
August 31, 2010 at 1:35 AM
A couple of things now that I’ve played with lxrng for a bit.
(hopefully it formats legibility)
first, your “git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git” repo for the kernel took my machine like 36 hours to process the whole thing, (not sure if my machine was trying to crunch something else at the same time) then it didn’t show the sub versions, eg 2.6.35.3. I believe http://lxr.linux.no/linux/ is using “git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6-stable.git” as their repo. I did e-mail them about it s couple of weeks ago, but haven’t heard anything yet. This repo only took my server about an hour or two to process with the lxr-genref and it has the sub versions. also I’d suggest running it with the nice command, especially if your server needs to do anything else.
second, I also used the lxrng to cross reference some Linux drivers and sdk stuff. since the driver didn’t have a public repo associated with it, I created a new git repo and added each version of the driver and sdk to the repo and tagged them with the version number. I’ve found it very useful so I though this would be a good place to share.
mkdir /home/web/lxrng/repos/random-drv
cd $_
git init
git add .
git commit -a -m “random-linux-driver 1.0″
git log
git tag v1.0
the command: “git tag” will show you all your current tags and “git status” will show your current repo status. (the status is most interesting before and after the git add .)
once you’re done you can add the new repo in your lxrng.conf and do your lxr-genxref and you’re good to go.
NOTE: if you forget the ‘v’ in front of the version number it won’t show up on the web page unless you chage the release_re value in the lxrng.conf for your new repo. also I believe the default value will cut off anything after a dash in the version number.
if you want the release candidates you’ll need to change the regex value for the release_re to allow dashes.
i guess the third thing is that just to make sure someone doesn’t get confused I’d change “./lxr-db-admin linux –init” to “thalib@mohamed:/home/web/lxrng$ ./lxr-db-admin linux –init” in part 3. I know it’s kind of obvious that you’re running the command from the command-line, it’s just everywhere else you show the prompt.
thanks again for working through this, I had fought lxrng for a couple of days and put it on the back burner, then I found your post and easily corrected my error in a matter of minutes.
August 31, 2010 at 10:42 AM
Hi
Thanks for your comment, my reply goes below.
1. regarding git clone took 36 hours, yes it will take such long time if your net connection is slow or some parallel download is going on.
2. I will try your suggestion of using linux-2.6-stable.git. but some time I will net the RC also so I used torvalds/linux-2.6.git.
3. Regarding “‘v’ in front of the version number”, I don’t have any idea. I will look into it.
4. Thanks for the correction friend, I have corrected it now.
I am glad that it helped you and thanks for sharing your recipes.
Regards,
Mohamed Thalib H
August 13, 2010 at 3:14 AM
How do you suggest updating the Lxrng data when a new version of the Linux kernel (or whichever repo) comes out? I was just thinking about a monthly or so cron job to update the git repository and the cross-reference data.
also in part three you reference your new git repo variable that doesn’t exist until part 4. this might confuse people. maybe change “‘repository’ => $ubootrepo,” to “‘repository’ => $gitrepo,” in step 3?
August 13, 2010 at 10:42 AM
Hi,
I have updated how to update lxrng database when newer version of linux arrives, in step 5.
Thanks for helping me finding the error. I have modified it and update the blog.
July 24, 2010 at 7:23 PM
[...] This post was mentioned on Twitter by Mohamed Thalib H, Mohamed Thalib H. Mohamed Thalib H said: How to setup LXR – Step by Step guide: http://wp.me/pyVsB-3E [...]