Optware

From DD-WRT Wiki

Revision as of 23:43, 20 January 2007 by Oleo (Talk | contribs)
Jump to: navigation, search

Contents

Introduction

With the latest DD-WRT v23 SP2, it is possible to install Optware packages from NSLU2-Linux project. This port of Optware is tailored for DD-WRT and extends the router to full feature linux if enough storage is provided for packages (jffs, mmc, USB?).

The main feature of this package is that it installs all required files under /opt with 600+ ported applications. With buildroot toolchain, system library independence is also added. This means that packages now depend on /opt/lib/uClibc.so and dynamic loader also in /opt/lib with separate /opt/etc/ld-opt.so.conf for custom .so cache. Applications are built with wchar and locale support which makes uClibc library somewhat greater than other uClibc packages, but still far smaller than glibc based aplications.

Buildroot toolchain features:

Basic preparation of /jffs is required (enable, clean). On WRT54GS there is room for some aplications. SD card (/mmc) may give enough space for many applications. For testing purposes also smbmount can be used. It should be noted that writable /jffs partition must be provided as /bin/ipkg script requires it to save some data there. If a non /jffs filesystem is available, it is recommended to unmount /jffs and then mount eg. mmc in place of /jffs.

Example scenario of installing some packages on WRT54GS

Preparing /jffs for /opt

First step is to prepare /opt mount point for Optware and create ipkg.conf file for packages download

~ # mkdir /jffs/opt
~ # mkdir /jffs/tmp
~ # mkdir /jffs/tmp/ipkg
~ # mount -o bind /jffs/opt /opt 
~ # mkdir /opt/etc
~ # vi /opt/etc/ipkg.conf

Edit and insert the following content

src optware http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable
dest root /jffs

Now we will point /bin/ipkg script to this config with setting environment variable

~ # export IPKG_CONF_DIR=/opt/etc

This is enough to retrieve list of available packages with

~ # ipkg update 
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/Packages ...
Connecting to ipkg.nslu2-linux.org[140.211.166.82]:80 
Packages             100% |**********************************************|   259 KB 00:00:00 ETA
Done.
Updated list of available packages in //jffs/usr/lib/ipkg/lists/optware

If nothing is reported after /bin/ipkg update then probably /etc/resolv.conf does not contain proper name server address like:

search local.lan
nameserver 192.168.1.1

Installing ipkg-opt and uclibc-opt

First we will install uClibc library required by all packages and better ipkg-opt packager. Installing package ipkg-opt will automatically install also uclibc-opt package.

~ # /bin/ipkg install ipkg-opt

Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/uclibc-opt_0.9.28-12_mipsel.ipk ...
Connecting to ipkg.nslu2-linux.org[140.211.166.82]:80
uclibc-opt_0.9.28-12 100% |******************************************|   832 KB 00:00:00 ETA
Done.
Unpacking uclibc-opt...Done.
Configuring uclibc-opt...Updating /opt/etc/ld.so.cache
Done.

Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ipkg-opt_0.99.163-9_
mipsel.ipk ...
Connecting to ipkg.nslu2-linux.org[140.211.166.82]:80
ipkg-opt_0.99.163-9_ 100% |******************************************| 75896    00:00:00 ETA
Done.
Unpacking ipkg-opt...Done.
Configuring ipkg-opt...Configuration file `/opt/etc/ipkg.conf'
 ==> File on system created by you or by a script.
 ==> File also in package provided by package maintainer.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions (if diff is installed)
 The default action is to keep your current version.
*** ipkg.conf (Y/I/N/O/D) [default=N] ? y
Done.

Note that we answered 'y' to install maintainer /opt/etc/ipkg.conf with the following contents:

#Uncomment the following line for native packages feed (if any)
#src/gz native http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/native/stable
src/gz optware http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable
dest /opt/ /

If anything goes wrong when installing ipkg-opt then I recommend erasing /jffs/jffs and /jffs/usr directories with

rm -rf /jffs/jffs
rm -rf /jffs/usr

and repeat /bin/ipkg update and /bin/ipkg install ipkg-opt

Installing ipkg-opt package will maintain all packages installed in /opt directory along with its /opt/lib/ipkg directory where /opt/bin/ipkg maintains its status files. There is also /opt/bin/ipkg-opt to save us from confusion. To update list of available packages issue

~ # /opt/bin/ipkg-opt update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/Packages.gz
Inflating http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/Packages.gz
Updated list of available packages in /opt/lib/ipkg/lists/optware
Successfully terminated.

List of available packages is provided with

~ # /opt/bin/ipkg-opt list

List of installed packages

~ # /opt/bin/ipkg-opt list_installed
Successfully terminated.

will not show ipkg-opt and uclibc-opt as status file is now located in /opt/lib/ipkg/status and not /jffs/jffs/usr/lib/ipkg/status . To correct this issue we will reinstall the package itself with

~ # /opt/bin/ipkg-opt install ipkg-opt
Installing ipkg-opt (0.99.163-9) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ipkg-opt_0.99.163-9_mipsel.ipk
package uclibc-opt suggests installing ipkg-opt
Installing uclibc-opt (0.9.28-12) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/uclibc-opt_0.9.28-12_mipsel.ipk
Configuring ipkg-opt
Configuring uclibc-opt
Updating /opt/etc/ld.so.cache
Successfully terminated.

With clean Optware install two packages should be listed

~ # /opt/bin/ipkg list_installed
ipkg-opt - 0.99.163-9 - The Itsy Package Manager
uclibc-opt - 0.9.28-12 - micro C library for embedded Linux systems
Successfully terminated.

From now on no files outside /opt tree will be required for maintaining Optware packages. I even recommend erasing spurious files with

# rm -rf /jffs/jffs /jffs/usr

Exporting IPKG_CONF_DIR is not neeeded any more.

/opt backup

Now we have prepared /opt for adding more packages. We can also create backup of the whole /opt tree in case of router upgrade, to give preinstalled /opt to a friend or just safety backup. We will create such backup with

~ # tar czf /tmp/opt-backup.tar.gz /opt

Destination for /tmp/opt-backup.tar.gz should be large enough! Beware that /tmp is RAM. Destination should be disk or remote smbfs partition. Extracting /opt backup could be:

cd /
rm -rf opt/*
tar xvzf /tmp/opt-backup.tar.gz

If /opt space is tight and no packages requiring libstdc++ will be used (like Amule) then removing this large shared library is recommended

 ~ # rm /opt/lib/libstdc\+*

will remove

 /opt/lib/libstdc++.so         /opt/lib/libstdc++.so.6       /opt/lib/libstdc++.so.6.0.8 

ipkg-opt downloads packages into /opt directory and erases them after installing. If you have enough free RAM and /jffs only then adding --tmp-dir /tmp is recommended instead of default /opt

Search PATH

Adding /opt/bin to search PATH will save us from tedious typing of /opt/bin for each command from /opt/bin. For running applications from /opt tree search path should be expanded with

~# export PATH=/opt/bin:/opt/sbin:$PATH 

Now we can install any package from a list of available packages.

~# ipkg-opt list | more

or less or this Quick list of Optware packages


Installing ncFTP client

We want to get list of packages containing name ftp

~ # /opt/bin/ipkg-opt list '*ftp*'
atftp - 0.7-6 - Advanced TFTP server and client
ftpcopy - 0.6.7-3 - A simple FTP client written to copy files or directories (recursively) from a FTP server.
ftpd-topfield - 0.6.10-2 - FTPD for the Topfield TF5000PVRt
ncftp - 3.2.0-2 - Nice command line FTP client
proftpd - 1.3.0a-1 - Highly configurable FTP server with SSL-TLS
tftp-hpa - 0.45-1 - A tftp package
tnftp - 20050625-1 - tnftp (formerly lukemftp) is what many users affectionately call the enhanced ftp client in NetBSD.
tnftpd - 20061217-2 - tnftpd (formerly lukemftpd) is a port of the NetBSD FTP server to other systems.
vsftpd - 2.0.1-7 - ftp daemon with an emphasis on speed and security
Successfully terminated.


For example we will install ncFTP client with:

~ # /opt/bin/ipkg  --tmp-dir /tmp install ncftp 
Installing ncftp (3.2.0-2) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ncftp_3.2.0-2_mipsel.ipk
Installing ncurses (5.6-1) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ncurses_5.6-1_mipsel.ipk
Configuring ncftp
Configuring ncurses
Successfully terminated.

and run it

~ # /opt/bin/ncftp ftp.si.debian.org 
NcFTP 3.2.0 (Aug 05, 2006) by Mike Gleason (http://www.NcFTP.com/contact/).

Copyright (c) 1992-2005 by Mike Gleason.
All rights reserved.

Connecting to 164.8.6.249...                                                                                        
CAMTP Debian mirror [ftp.si.debian.org]
Logging in...                                                                                                       
Anonymous access granted, restrictions apply.
Logged in to ftp.si.debian.org.                                                                                     
ncftp / > ls
backports/      debian-cd/      debian-sl/      openoffice/
debian/         debian-non-US/  favicon.ico     robots.txt
ncftp / > bye


You have not saved a bookmark for this site.

Would you like to save a bookmark to:
        ftp://ftp.si.debian.org

Save? (yes/no) yes
Enter a name for this bookmark, or hit enter for "si": debian
Bookmark "debian" saved.

Always check available size of your /opt partition with

~ # df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/root                 1984      1984         0 100% /
/dev/mtdblock/4           5248      3860      1388  74% /jffs
/dev/mtdblock/4           5248      3860      1388  74% /opt

Remove unneeded packages with

~ # /opt/bin/ipkg remove ncftp
Removing package ncftp from /opt/...
Successfully terminated.

You will notice that ncurses as depended package will not get removed. To correct this issue

~ # /opt/bin/ipkg remove ncurses
Removing package ncurses from /opt/...
Successfully terminated.

unless some other package require this package.

Installing torrent client

Of course there are many more packages which should run on router without a problem. All binaries have built in /opt/lib search path and there should be no problem with libraries clash like problems with OpenWRT packages. All packages have builtin package dependencies and will install required libraries automagically. Lets show this on installing torrent clients:

~ # /opt/bin/ipkg -t /tmp install enhanced-ctorrent 
Installing enhanced-ctorrent (dnh2.2-6) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/enhanced-ctorrent_dnh2.2-6_mipsel.ipk
Installing libuclibc++ (0.2.2-4) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/libuclibc++_0.2.2-4_mipsel.ipk
Installing openssl (0.9.7l-3) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/openssl_0.9.7l-3_mipsel.ipk
 Configuring enhanced-ctorrent
Configuring libuclibc++
Configuring openssl
Successfully terminated.


We will not show running example for this clients as normally huge amount of storrage is required.

Another torrent client with WWW interface is Transmission. Installing is simple as

~ # /opt/bin/ipkg install transmission
Installing transmission (0.6+r1378-5) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/transmission_0.6+r1378-5_mipsel.ipk
package transmission suggests installing gnuplot
package transmission suggests installing logrotate
Installing openssl (0.9.7l-3) to /opt/...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/openssl_0.9.7l-3_mipsel.ipk
Configuring openssl
Configuring transmission
Please note that /opt/etc/init.d/S80busybox_httpd is installed for
running CGI interface at http://localhost:8008/cgi-bin/transmission.cgi
See instructions for daemon and WWW interface in /opt/share/doc/transmission
Configure daemon with /opt/etc/transmission.conf
Successfully terminated.

Besides simple command line transmissioncli client like enhanced-ctorrent, there is also possibility to run several torrents in daemon mode (background mode) with WWW page as interface watching daemon state and controlling its work.

After creating directories with

~ # mkdir /opt/torrent
~ # mkdir /opt/torrent/source
~ # mkdir /opt/torrent/work
~ # mkdir /opt/torrent/target

we will configure transmisison itself by editing /opt/etc/transmisison.conf by using

~ # sed -i -e 's|/tmp/harddisk|/opt|' /opt/etc/transmission.conf

to change default path from /tmp/harddisk to /opt. Also user should be changed from admin to root. After changing config transmission_watchdog should be run every half an hour with cron. We will start it manually and check if it is running

~ # /opt/sbin/transmission_watchdog
~ # ps |grep transmission
10621 root        808 S   transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var
10624 root        808 S   transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var
10625 root        808 S   transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var
10626 root        808 R   transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var

We will configure http server to show the startup page at port 8008. Although dd-wrt SP2 uses /opt/sbin/httpd for its WEB interface, this httpd server cannot be reused as it has serioius problems running CGI programs and showing index.html files.

For public trackers CGI interface provides 'Fetch' button which downloads .torrent file into /opt/torrent/source directory. Private trackers use passkey which means that .torrent file should be saved with browser itself.

After seeing .torrent file int Queued list, one can press Watchdog button to force queue processing. This will move .toorent file from source into work directory and notify daemon for the change to start processing the file. After download finishes, transmission_watchdog will automaticly move torrent from work to target. Meanwhile, user can pres Update button to see status of active torrents.

Besides transmissiond two other daemons are required for normal queue processing. Both are provided by dd-wrt if enabled. Cron should regulary run /opt/sbin/transmisison_watchdog with sample /opt/etc/crontab

*/30 * * * * run nice /opt/sbin/transmission_watchdog

Syslog will collect cumulative reports from transmisisond and CGI will show it when Log or Update is pressed. If gnuplot is installed then also transfer graph is showed when Log is pressed.

Installing Throtling HTTP server

For our purposes thttpd WEB server will handle most requirements like browsing directories and runing CGI program located at /opt/share/www/cgi-bin/transmission.cgi

~ # /opt/bin/ipkg install thttpd
Installing thttpd (2.25b-4) to /opt/...
Downloading  http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/thttpd_2.25b-4_mipsel.ipk
Configuring thttpd


To complete the installation, you must edit /opt/etc/thttpd.conf,
and then run /opt/etc/init.d/S80thttpd to start thttpd.

Successfully terminated.

Edit /opt/etc/thttpd.conf with vi (or nano) and add cgipat, change user to root, change port to 8008 to see the following contents:

dir=/opt/share/www
port=8008
user=root
nochroot
nosymlink
novhost
logfile=/var/log/thttpd.log
pidfile=/var/run/thttpd.pid
cgipat=cgi-bin/*

Manually start thttpd with

~ # /opt/etc/init.d/S80thttpd start

After that you will be able to see default index.html page at http://192.168.1.1:8008/ assuming that your router is at 192.168.1.1 . There is also link to CGI page http://192.168.1.1:8008/cgi-bin/transmission.cgi which is preaty empty with several buttons for controlling daemon.

Remarks

  • DD-WRT provided /bin/ipkg is not full feature ipkg but it can do the job for installing ipkg-opt
  • Using samba share for /opt is not easy as there is no notion for symbolic links on SMB. This is most problematic for symbolic links on shared libraries. Rewriting "ln" to act as a script and put it on the head of search path could be used to copy instead of link. Possibility of mounting filesystem in a file as a loop device on a samba share is not yet verified.
  • DD-WRT kernel lacks many "server" features like NFS, USB support, ... Provided samba, mysql, thttp-php and other applications can do the job if enough space is provided. This means that external storage is required for memory hungry applications. Experience from nslu2 and Oleg firmware showed that 512MB swap space is recommended. And this can only be provided with external USB harddisk.
  • Optware for DD-WRT is beta-stage feature. Report problems and suggestions in DD-WRT forum under Optware.
  • For instuctions on some other packages look at http://www.nslu2-linux.org/wiki/Unslung/Packages
  • Package feed will eventually change to official NSLU2 package feeds
  • Optware package tracking and repository access
  • There is also posibility to install Optware packages under Oleg, X-wrt or OpenWRT Whiterussian firmware. See instructions on http://www.nslu2-linux.org/wiki/FAQ/Optware-uClibcBuild

Oleo