Optware
From DD-WRT Wiki
Revision as of 11:10, 20 March 2008 (edit) Soulstace (Talk | contribs) (Undo revision 19687 by Eowyn (Talk)) ← Previous diff |
Revision as of 10:59, 28 May 2008 (edit) (undo) Knasher (Talk | contribs) m (Update script to reflect version change) Next diff → |
||
Line 44: | Line 44: | ||
possible network problems and /opt directory availability. To install (or reinstall) issue the following | possible network problems and /opt directory availability. To install (or reinstall) issue the following | ||
two commands: | two commands: | ||
- | wget [http://pastebin.ca// | + | wget [http://pastebin.ca//1031954 http://pastebin.ca/raw/1031954] -O - | tr -d '\r' > /tmp/optware-install.sh |
sh /tmp/optware-install.sh | sh /tmp/optware-install.sh | ||
Output should look like: | Output should look like: |
Revision as of 10:59, 28 May 2008
Contents |
Introduction
With the latest DD-WRT (v23 SP2 and v24 SP2), it is possible to install Optware packages from NSLU2-Linux project. This port of Optware is tailored for DD-WRT and extends the router to full feature linux if enough storage (jffs, SD/MMC Card, USB) is provided for packages.
The main feature of this package is that it installs all required files under /opt with 1000 ported applications. With buildroot toolchain, system library independence is also added. This means that packages now depend on /opt/lib/uClibc.so and dynamic loader also in /opt/lib with separate /opt/etc/ld-opt.so.conf for custom .so cache. Applications are built with wchar and locale support which makes uClibc library somewhat greater than other uClibc packages, but still far smaller than glibc based aplications.
Buildroot toolchain features:
- GCC 4.1.1
- uClibc 0.9.28 with many patches
- uClibc++ 0.2.2
- Native POSIX Threads Library (NPTL) Support for uClibc
Some packages (perl, python, Transmission, buildroot, ...) are so memory hungry that swap is needed for normal operation. Buildroot package or optware-devel meta package provides complete development toolchain for compiling programs on router itself with gcc, gdb, make, g++ , svn, ...
Basic preparation of /jffs is required (enable, clean). On WRT54GS there is room for some aplications. SD card (/mmc) may give enough space for many applications. For testing purposes also smbmount can be used. If a non /jffs filesystem is available, it is recommended to unmount /jffs and then mount eg. mmc in place of /jffs.
Prerequisites
As always here are two prerequisites.
- Hardware
- Recommended are routers with USB support (Asus WL-500g Deluxe|Premium, Linksys WRTSL54GS).
- WRT54g routers with hardware modification for MMC/SD card
- 32MB of RAM may not be enough for some application and thus swap is recommended
- USB hard disk for storage and swap or USB key without swap
- Routers with 8MB flash can also fit some applications in /jffs without the need for swap (e.g. WRT54GS v1.1)
- Software
- DD-WRT v23 SP2 mini or standard when MMC used
- You should be able to use telnet and the command line.
Example scenario of installing some packages on WRT54GS (v1.0 .. v3.0)
Preparing /jffs for /opt
Prerequisite for optware is writable /opt partition. Required size depends on packages installed. WRT54GS with 8MB flash and DD-WRT SP2 mini leaves 5248k blocks for /jffs which can be binded to /opt.
~ # mkdir /jffs/opt ~ # mount -o bind /jffs/opt /opt
Installing ipkg-opt and uclibc-opt
First we will install uClibc library required by all packages and better ipkg-opt packager. Recommended way for installing basic two packages is with installation script that will also check for possible network problems and /opt directory availability. To install (or reinstall) issue the following two commands:
wget http://pastebin.ca/raw/1031954 -O - | tr -d '\r' > /tmp/optware-install.sh sh /tmp/optware-install.sh
Output should look like:
Checking system config ... Using 192.168.1.1 as default gateway. Using the following nameserver(s): nameserver 192.168.1.30 Warning: local nameserver is different than gateway! Check config or enter: sed -i s/192.168.*/192.168.1.1/ /tmp/resolv.conf to correct this. Installing package uclibc-opt_0.9.28-13_mipsel.ipk ... Connecting to ipkg.nslu2-linux.org[140.211.166.82]:80 uclibc-opt_0.9.28-12 100% |***********************************************| 832 KB 00:00:00 ETA Updating /opt/etc/ld.so.cache /opt/sbin/ldconfig: can't create /opt/etc/ld.so.cache~ (No such file or directory) Installing package ipkg-opt_0.99.163-9_mipsel.ipk ... Connecting to ipkg.nslu2-linux.org[140.211.166.82]:80 ipkg-opt_0.99.163-9_ 100% |***********************************************| 75896 00:00:00 ETA Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/Packages.gz Inflating http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/Packages.gz Updated list of available packages in /opt/lib/ipkg/lists/optware Successfully terminated. Installing uclibc-opt (0.9.28-12) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/uclibc-opt_0.9.28-12_mipsel.ipk package uclibc-opt suggests installing ipkg-opt Configuring uclibc-opt Updating /opt/etc/ld.so.cache Successfully terminated. Installing ipkg-opt (0.99.163-9) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ipkg-opt_0.99.163-9_mipsel.ipk Configuring ipkg-opt Successfully terminated.
Besides pre installing fundamental packages, there are also manual installation instructions. This are now somewhat deprecated due to large number of user miss-understandings.
/opt backup
Now we have prepared /opt for adding more packages. We can also create backup of the whole /opt tree in case of router upgrade, to give preinstalled /opt to a friend or just safety backup. We will create such backup with
~ # tar czf /tmp/opt-backup.tar.gz /opt
Destination for /tmp/opt-backup.tar.gz should be large enough! Beware that /tmp is RAM. Destination should be disk or remote smbfs partition. Extracting /opt backup could be:
cd / rm -rf opt/* tar xvzf /tmp/opt-backup.tar.gz
If /opt space is tight and no packages requiring libstdc++ will be used (like Amule) then removing this large shared library is recommended
~ # rm /opt/lib/libstdc\*
will remove
/opt/lib/libstdc++.so /opt/lib/libstdc++.so.6 /opt/lib/libstdc++.so.6.0.8
ipkg-opt downloads packages into /opt directory and erases them after installing. If you have enough free RAM and /jffs only then adding --tmp-dir /tmp is recommended instead of default /opt
Search PATH
Adding /opt/bin to search PATH will save us from tedious typing of /opt/bin for each command from /opt/bin. For running applications from /opt tree search path should be expanded with
~# export PATH=/opt/bin:/opt/sbin:$PATH
Now we can install any package from a list of available packages.
~# ipkg-opt list | more
or less or this Quick list of Optware packages DD-WRT sets LD_LIBRARY_PATH in /etc/profile that can cause library loading clash. Unset this variable with
unset LD_LIBRARY_PATH
Installing ncFTP client
We want to get list of packages containing name ftp
~ # /opt/bin/ipkg-opt list '*ftp*' atftp - 0.7-6 - Advanced TFTP server and client ftpcopy - 0.6.7-3 - A simple FTP client written to copy files or directories (recursively) from a FTP server. ftpd-topfield - 0.6.10-2 - FTPD for the Topfield TF5000PVRt ncftp - 3.2.0-2 - Nice command line FTP client proftpd - 1.3.0a-1 - Highly configurable FTP server with SSL-TLS tftp-hpa - 0.45-1 - A tftp package tnftp - 20050625-1 - tnftp (formerly lukemftp) is what many users affectionately call the enhanced ftp client in NetBSD. tnftpd - 20061217-2 - tnftpd (formerly lukemftpd) is a port of the NetBSD FTP server to other systems. vsftpd - 2.0.1-7 - ftp daemon with an emphasis on speed and security Successfully terminated.
For example we will install ncFTP client with:
~ # /opt/bin/ipkg --tmp-dir /tmp install ncftp Installing ncftp (3.2.0-2) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ncftp_3.2.0-2_mipsel.ipk Installing ncurses (5.6-1) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/ncurses_5.6-1_mipsel.ipk Configuring ncftp Configuring ncurses Successfully terminated.
and run it
~ # /opt/bin/ncftp ftp.si.debian.org NcFTP 3.2.0 (Aug 05, 2006) by Mike Gleason (http://www.NcFTP.com/contact/). Copyright (c) 1992-2005 by Mike Gleason. All rights reserved. Connecting to 164.8.6.249... CAMTP Debian mirror [ftp.si.debian.org] Logging in... Anonymous access granted, restrictions apply. Logged in to ftp.si.debian.org. ncftp / > ls backports/ debian-cd/ debian-sl/ openoffice/ debian/ debian-non-US/ favicon.ico robots.txt ncftp / > bye You have not saved a bookmark for this site. Would you like to save a bookmark to: ftp://ftp.si.debian.org Save? (yes/no) yes Enter a name for this bookmark, or hit enter for "si": debian Bookmark "debian" saved.
Always check available size of your /opt partition with
~ # df Filesystem 1k-blocks Used Available Use% Mounted on /dev/root 1984 1984 0 100% / /dev/mtdblock/4 5248 3860 1388 74% /jffs /dev/mtdblock/4 5248 3860 1388 74% /opt
Remove unneeded packages with
~ # /opt/bin/ipkg remove ncftp Removing package ncftp from /opt/... Successfully terminated.
You will notice that ncurses as depended package will not get removed. To correct this issue
~ # /opt/bin/ipkg remove ncurses Removing package ncurses from /opt/... Successfully terminated.
unless some other package require this package.
Installing torrent client
Of course there are many more packages which should run on router without a problem. All binaries have built in /opt/lib search path and there should be no problem with libraries clash like problems with OpenWRT packages. All packages have builtin package dependencies and will install required libraries automagically. Lets show this on installing torrent clients:
~ # /opt/bin/ipkg -t /tmp install enhanced-ctorrent Installing enhanced-ctorrent (dnh2.2-6) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/enhanced-ctorrent_dnh2.2-6_mipsel.ipk Installing libuclibc (0.2.2-4) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/libuclibc _0.2.2-4_mipsel.ipk Installing openssl (0.9.7l-3) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/openssl_0.9.7l-3_mipsel.ipk Configuring enhanced-ctorrent Configuring libuclibc Configuring openssl Successfully terminated.
We will not show running example for this clients as normally huge amount of storrage is required.
Another torrent client with WWW interface is Transmission. Installing is simple as
~ # /opt/bin/ipkg install transmission Installing transmission (0.6+r1378-5) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/transmission_0.6+r1378-5_mipsel.ipk package transmission suggests installing gnuplot package transmission suggests installing logrotate Installing openssl (0.9.7l-3) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/openssl_0.9.7l-3_mipsel.ipk Configuring openssl Configuring transmission Please note that /opt/etc/init.d/S80busybox_httpd is installed for running CGI interface at http://localhost:8008/cgi-bin/transmission.cgi See instructions for daemon and WWW interface in /opt/share/doc/transmission Configure daemon with /opt/etc/transmission.conf Successfully terminated.
Besides simple command line transmissioncli client like enhanced-ctorrent, there is also possibility to run several torrents in daemon mode (background mode) with WWW page as interface watching daemon state and controlling its work.
After creating directories with
~ # mkdir /opt/torrent ~ # mkdir /opt/torrent/source ~ # mkdir /opt/torrent/work ~ # mkdir /opt/torrent/target
we will configure transmisison itself by editing /opt/etc/transmisison.conf by using
~ # sed -i -e 's|/tmp/harddisk|/opt|' /opt/etc/transmission.conf
to change default path from /tmp/harddisk to /opt. Also user should be changed from admin to root. After changing config transmission_watchdog should be run every half an hour with cron. We will start it manually and check if it is running
~ # /opt/sbin/transmission_watchdog ~ # ps |grep transmission 10621 root 808 S transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var 10624 root 808 S transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var 10625 root 808 S transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var 10626 root 808 R transmissiond -p 65534 -w 300 -u -1 -d -1 -i /opt/var
We will configure http server to show the startup page at port 8008. Although dd-wrt SP2 uses /opt/sbin/httpd for its WEB interface, this httpd server cannot be reused as it has serioius problems running CGI programs and showing index.html files.
For public trackers CGI interface provides 'Fetch' button which downloads .torrent file into /opt/torrent/source directory. Private trackers use passkey which means that .torrent file should be saved with browser itself.
After seeing .torrent file int Queued list, one can press Watchdog button to force queue processing. This will move .toorent file from source into work directory and notify daemon for the change to start processing the file. After download finishes, transmission_watchdog will automaticly move torrent from work to target. Meanwhile, user can pres Update button to see status of active torrents.
Besides transmissiond two other daemons are required for normal queue processing. Both are provided by dd-wrt if enabled. Cron is enabled under Admin control panel. Syslogd is enabled under Services tab.
Cron should regulary run /opt/sbin/transmisison_watchdog with sample /opt/etc/crontab
*/30 * * * * root /opt/sbin/transmission_watchdog
Visit CRON page to see where this line should be included. I recommend
cat /opt/etc/crontab >> /tmp/crontab
at startup,
Syslog will collect cumulative reports from transmisisond and CGI will show it when Log or Update is pressed. If gnuplot is installed then also transfer graph is showed when Log is pressed. SYSLOG line in /opt/etc/transmission.conf must be changed to
SYSLOG=/var/log/messages
to be able to see Log messages from daemon when Log button is pressed.
Opening default listening port 65534 for Transmission is another required task which can be solved by DD-WRT control panel or adding
/usr/sbin/iptables -I INPUT 1 -p tcp --dport 65534 -j logaccept
to startup file.
Installing Throttling HTTP server
For our purposes thttpd WEB server will handle most requirements like browsing directories and runing CGI program located at /opt/share/www/cgi-bin/transmission.cgi
~ # /opt/bin/ipkg install thttpd Installing thttpd (2.25b-4) to /opt/... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ddwrt/cross/stable/thttpd_2.25b-4_mipsel.ipk Configuring thttpd To complete the installation, you must edit /opt/etc/thttpd.conf, and then run /opt/etc/init.d/S80thttpd to start thttpd. Successfully terminated.
Edit /opt/etc/thttpd.conf with vi (or nano) and add cgipat, change user to root, change port to 8008 to see the following contents:
dir=/opt/share/www port=8008 user=root nochroot nosymlink novhost logfile=/var/log/thttpd.log pidfile=/var/run/thttpd.pid cgipat=cgi-bin/*
Manually start thttpd with
~ # /opt/etc/init.d/S80thttpd start
After that you will be able to see default index.html page at http://192.168.1.1:8008/ assuming that your router is at 192.168.1.1 . There is also link to CGI page http://192.168.1.1:8008/cgi-bin/transmission.cgi which is preaty empty with several buttons for controlling daemon.
Installing Samba2
It looks like Samba 3 is too big for all "swapless" routers. For this reason I have prepared Samba 2 packages with Samba Web Admin Tool (SWAT) which will ease config and admin for samba newcomers. SWAT interface looks like:
Installation and running samba 2 is done out of the box by the following commands:
/opt/bin/ipkg update /opt/bin/ipkg remove samba /opt/bin/ipkg install samba2 /opt/bin/ipkg install xinetd /opt/etc/init.d/S10xinetd start /opt/etc/init.d/S80samba start
Point your browser to http://192.168.1.1:901/ to administer samba. Installation of xinetd is optional and it is not required if one wants to manually edit /opt/etc/samba/smb.conf
The problem that dd-wrt does not include user nobody at startup can be overcome with adding this at startup with /opt/etc/init.d/S80samba script like:
#!/bin/sh [ -d /tmp/samba ] || mkdir /tmp/samba grep -q nobody /etc/passwd || echo "nobody:*:65534:65534:nobody:/var:/bin/false" >> /etc/passwd killall smbd killall nmbd /opt/sbin/nmbd -D /opt/sbin/smbd -D
Another long term problem that could happen when running samba on FLASH memory like jffs, mmc or USB stick is FLASH wear. Such use could destroy FLASH if log files and browsing databases are constantly updated. Use of /tmp partition is recommended for such use. Amend startup script with:
echo "Starting smbd:" if [ ! -d /tmp/samba ]; then mkdir /tmp/samba mkdir /tmp/samba/log rm -rf /opt/var/log/samba ln -s /tmp/samba/log /opt/var/log/samba rm -rf /opt/var/samba ln -s /tmp/samba /opt/var/samba fi
Running at startup
DD-WRT provides user startup files in /jffs/etc/config . To create optware startup file do
~ # mkdir /jffs/etc ~ # mkdir /jffs/etc/config ~ # vi /jffs/etc/config/optware.startup
with the following contents
#!/bin/sh mount -o bind /jffs/opt /opt unset LD_LIBRARY_PATH unset LD_PRELOAD [ -e /opt/etc/profile ] && mount -o bind /opt/etc/profile /etc/profile if [ -d /opt/etc/init.d ]; then for f in /opt/etc/init.d/S* ; do [ -x $f ] && $f start done fi
and mark it as executable with
~ # chmod +x /jffs/etc/config/optware.startup
After reboot this file will be executed. I recommed first trying it manually
~ # /jffs/etc/config/optware.startup
Creating /opt/etc/profile with the following contents
export PATH=/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin export PS1='\u@\h:\w\$ ' [ -x /opt/bin/less ] || alias less=more [ -x /opt/bin/vim ] || alias vim=vi arp() { cat /proc/net/arp; } ldd() { LD_TRACE_LOADED_OBJECTS=1 $*; } reboot() { killall nmbd killall smbd killall xinetd killall thttpd sleep 1 umount /opt /sbin/reboot }
is also recommended as this will unset LD_LIBRARY_PATH provided by stock /etc/profile
Startup for external USB Disk
The following script is recommended when using external USB hard disk. The main feature of this script is prevention of disk corruption and automatic replacement. It also shows how LD_LIBRARY_PATH can be used to compensate uClibc incompatibility between dd-wrt /lib and OpenWRT /jffs/lib . With dd-wrt.V24 the difference between system and OpenWRT Whiterussian R6 uClibc is so big, that running even simple programs causes segmentation fault. Optware also uses its own uClibc that resides in /opt/lib but all programs have built-in rpath to /opt/lib. This means that it is simpler to use Optware packages under DD-WRT than OpenWRT packages. But the following script relies on two OpenWRT packages just to provide filesystem checking and enabling swap at startup. Besides reguired kernel modules, two disk related packages were installed: e2fsprogs and swap-utils. All others packages could be used from Optware.
#!/bin/sh # /jffs/etc/config/optware.startup script for USB disk support # Recommended ext3 partitions: # /dev/discs/disc0/part1 for /opt 1GB # /dev/discs/disc0/part2 for swap 512MB # /dev/discs/disc0/part3 /mmc for user files aka MultiMediaCenter # Tune partition checks to max 10 counts disregarding last check time # tune2fs -i 0 -c 10 /dev/discs/disc0/part1 # tune2fs -i 0 -c 10 /dev/discs/disc0/part3 # Required OpenWrt packages: /bin/ipkg install swap-utils e2fsprogs ( unset LD_PRELOAD export PATH=/bin:/usr/bin:/sbin:/usr/sbin export LD_LIBRARY_PATH=/lib:/usr/lib echo "Inserting modules for USB disk support..." insmod /jffs/lib/modules/2.4.30/usbcore.o insmod /jffs/lib/modules/2.4.30/ehci-hcd.o insmod /jffs/lib/modules/2.4.30/scsi_mod.o insmod /jffs/lib/modules/2.4.30/usb-storage.o insmod /jffs/lib/modules/2.4.30/sd_mod.o insmod /jffs/lib/modules/2.4.30/jbd.o insmod /jffs/lib/modules/2.4.30/ext3.o echo "Waiting for modules to initialize disk access..." sleep 20 # assure correct paths when running OpenWRT programs like e2fsck export LD_LIBRARY_PATH=/jffs/lib:/jffs/usr/lib export PATH=/jffs/bin:/jffs/sbin:/jffs/usr/sbin:/jffs/usr/bin:/bin:/sbin echo "Activating swap..." swapon /dev/discs/disc0/part2 echo "Checking optware partition..." e2fsck -p /dev/discs/disc0/part1 echo "Checking user-data partition" e2fsck -p /dev/discs/disc0/part3 echo "Mountint disk partitions..." export LD_LIBRARY_PATH=/lib:/usr/lib mount -t ext3 -o noatime /dev/discs/disc0/part1 /opt mount -t ext3 -o noatime /dev/discs/disc0/part3 /mmc if [ -e /opt/etc/profile ]; then cp /opt/etc/profile /tmp mount -o bind /tmp/profile /etc/profile fi # provide Optware search paths unset LD_LIBRARY_PATH export PATH=/opt/bin:/opt/sbin:/bin:/sbin:/usr/sbin:/usr/bin echo "Starting Optware programs..." if [ -d /opt/etc/init.d ]; then for f in /opt/etc/init.d/S* ; do [ -x $f ] && $f start done fi ) > /tmp/optware.log 2>&1
Forcing filesystem recheck on next boot could be marked with
tune2fs -C 100 /dev/discs/disc0/part1
Although ext3 filesystem uses journaling that prevents major corruption and faster recover during crash, it is recommended that one writes its shutdown scrtipt as alias for /etc/profile that kills all optware daemons, umounts filesystems and then issue /sbin/reboot. This is what we call clean shutdown.
Known bugs
ngrep doesn't work
ngrep is compiled with IPV6 support and ddwrt doesn't provide IPV6 support.
If ngrep is recompiled without IPV6 support, it works ok after adding the nobody user to /etc/passwd
# grep nobody /etc/passwd nobody:x:99:99:Nobody:/:
Remarks
- DD-WRT provided /bin/ipkg is not full feature ipkg but it can do the job for installing ipkg-opt. It could also be used for installing other packages, but this is not recommended any more.
- Using samba share for /opt is not easy as there is no notion for symbolic links on SMB. This is most problematic for symbolic links on shared libraries. Rewriting "ln" to act as a script and put it on the head of search path could be used to copy instead of link. See this dd-wrt forum topic for info on issueof relinking. Possibility of mounting filesystem in a file as a loop device on a samba share is not yet verified.
- DD-WRT kernel lacks many "server" features like NFS, NBD support, ... Provided samba, mysql, thttp-php and other applications can do the job if enough space is provided. This means that external storage is required for memory hungry applications. Experience from nslu2 and Oleg firmware showed that 512MB swap space is recommended if external USB hard disk is available. See USB page for instructions how to initialize USB kernel modules and harddisk.
- Do not use VFAT filesystem for /opt. Use ext2 or ext3 instead. Alternatively one can use loop device to mount filesystem in file that resides on VFAT. See http://www.nslu2-linux.org/wiki/FAQ/Optware-uClibcBuild for example on doing this on USB stick.
- Optware for DD-WRT is beta-stage feature. Report problems and suggestions in DD-WRT forum under Optware.
- For instuctions on some other packages look at http://www.nslu2-linux.org/wiki/Unslung/Packages
- Optware package tracking and repository access
- There is also posibility to install Optware packages under Oleg, X-wrt or OpenWRT Whiterussian firmware. See instructions on http://www.nslu2-linux.org/wiki/FAQ/Optware-uClibcBuild