Journalling Flash File System

From DD-WRT Wiki

(Difference between revisions)
Jump to: navigation, search
Revision as of 23:39, 21 May 2008 (edit)
S2s2 (Talk | contribs)
(Second/third edition edit. First edition dumped when ff crashed. Using Opera!! yeah..)
← Previous diff
Revision as of 09:58, 22 May 2008 (edit) (undo)
S2s2 (Talk | contribs)
m (Looking more at the CIFS section)
Next diff →
Line 96: Line 96:
* Content actually saved remotely, on * Content actually saved remotely, on
** NAS (Network Attached Storage) ** NAS (Network Attached Storage)
-** Server (PC or some type of file server)<br>'''Costs:'''+** Server (PC or some type of file server)
-*** Equipment: Requires existing NAS or PC Server<br>Setting up a NAS or Server only for router maybe not economical.+'''Costs:'''
-*** Electricity: if entire Server is only serving router+* ! Setting up a NAS or Server only for router maybe not economical.
-*** Availability: CIFS share must be available+** Equipment Costs: Requires existing NAS or PC Server
-**** 24/7, or whenever the DD-WRT Device is on.+** Electricity Costs: if entire Server is only serving router
-**** Storing /jffs on quality equipment is required (i.e. Raid, Power Backup)<br>Remote storage down means /jffs down, could mean DD-WRT Device down (increases the weak points).+* Availability Demands: CIFS share must be available
-*** Security: CIFS is limited to Microsoft's SMB protocol, offering up to only 128-bit security.+** 24/7, or whenever the DD-WRT Device is on, and
-*** LAN Connection Required: NAS cannot be wirelessly attached to DD-WRT Device.+** Storing JFFS Share on quality equipment is required (i.e. Raid, Power Backup)
-*** Hassle: Using Microsoft Windows as the ''jffs'' Server+*** ! Remote storage down means /jffs down, could mean DD-WRT Device down (increases the weak points).
-**** Symbolic links in ''/jffs'' may not work.+* Security Limitations: CIFS is limited to Microsoft's SMB protocol, offering up to only 128-bit security.
-**** Rename (mv) commands on the ''/jffs'' mount may not work.+** Poor Microsoft Protocol Security is not a problem if the LAN is physically and electronically protected from outside or inside intrusion (i.e. Guards, Guns, Locks, Dogs, Mines, Cameras, Firewalls)
 +* LAN Connection Required: NAS cannot be wirelessly attached to DD-WRT Device.
 +* Hassle: Using Microsoft Windows as the ''jffs'' Server
 +** ! Symbolic links in ''/jffs'' may not work.
 +** ! Rename (mv) commands on the ''/jffs'' mount may not work.
'''Most people only own an (inexpensive) Windows PC, and it is ill-advised to use a common Desktop PC as a 24/7 Server.''' '''Most people only own an (inexpensive) Windows PC, and it is ill-advised to use a common Desktop PC as a 24/7 Server.'''
External Links:<br> External Links:<br>
-See [http://wrt-wiki.bsr-clan.de/index.php?title=Jffs.<br>+See http://wrt-wiki.bsr-clan.de/index.php?title=Jffs.<br>
-Linux Newbies: [http://wrt-wiki.bsr-clan.de/index.php?title=Jffs_sharing_in_a_Linux_server.<br>+Linux Newbies: http://wrt-wiki.bsr-clan.de/index.php?title=Jffs_sharing_in_a_Linux_server.<br>
-The following examples may be out of date:+'''The following non-updated (as of 20080520) examples may be out of date:'''
-===(Depricated) Use a Startup Script: Samba Example 1===+* CIFS was broken on many builds this update author tried. Although other people have reported it working - It may be a limitation of ''BusyBox'' CIFS when using NTLMv2 and 128-bit encryption.
-Read first, howto use [[Startup Scripts]]. Then place the following into your startup script to mount the JFFS partition:+* (Mostly) only structural and spelling mistakes were corrected below (20080522)
 +===Use a Startup Script: Example 1===
 +# Read how to use [[Startup Scripts]]. Then place the following into your startup script to mount the JFFS partition:
- umount /jffs+ umount /jffs #unmounts existing /jffs
- smbmount //<server's IP>/jffs /jffs -o username=<your username>,password=<your password>+ mount.cifs //<server's IP>/jffs /jffs -o username=<the username>,password=<the password>
 + nvram set sys_enable_jffs2=1 # fake-enable JFFS. Enable but do not commit (no format).
-*NOTE: username is guest if hosted on Windows XP & 'Simple File Sharing' is enabled.+*NOTE: username is ''Guest'' when hosted on Windows XP & 'Simple File Sharing' is enabled - This is the default on Windows XP Home.
-===(Depricated) Use Samba Startup Scripts: Example 2===+===Use Startup Scripts: Example 2===
-This method uses 2 scripts: The first will be started from /tmp/smbshare, mount the samba share under /jffs and start the other script. The second will be started from /jffs and unmount /tmp/sambashare.+This method uses 2 scripts: The first will be started from /tmp/smbshare & mount the samba share under /jffs and start the other script. The second will be started from /jffs and unmount /tmp/sambashare.
-Now here are instruction on how to do so:+Instructions:
-1) disable JFFS in the web GUI if you had enabled it<br>+# JFFS is 'fake-enabled' in the script.
-2) enable [[Samba Filesystem|samba]] (see also [[Jffs_sharing_in_a_Linux_server]]).<br>+#* With the JFFS NVRAM variable set to true (but not committed!), and the ''/jffs'' folder present, [[ipkg]] works.
 +# Enable [[CIFS]] (see also [[Jffs_sharing_in_a_Linux_server]]).
 +# Create a script called "startup.smb.sh" on the share:
-3) create a script called "startup.smb.sh" on the share:<br> 
#!/bin/sh #!/bin/sh
Line 145: Line 153:
#where to log what is happening #where to log what is happening
export SMBLOG=/tmp/smblog export SMBLOG=/tmp/smblog
- +
- +
####### ok, here we go... ;) ####### ####### ok, here we go... ;) #######
echo "" > $SMBLOG echo "" > $SMBLOG
cd / cd /
- #make sure JFFS is unmounted. Its recommended to run without JFFS enabled though.+ #make sure JFFS is unmounted.
echo "umounting /jffs if nescessary..." >> $SMBLOG echo "umounting /jffs if nescessary..." >> $SMBLOG
umount /jffs >> $SMBLOG 2>&1 umount /jffs >> $SMBLOG 2>&1
Line 157: Line 164:
#mount our smbshare at /jffs #mount our smbshare at /jffs
echo "mounting $SMBSHARE at /jffs..." >> $SMBLOG echo "mounting $SMBSHARE at /jffs..." >> $SMBLOG
- smbmount $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1+ #Depricated: smbmount $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1
- #smbmount ist not working in v23 SP3, so i changed the line above:+ mount.cifs $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1
- #mount.cifs $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1+
#hand over startup to the JFFS part #hand over startup to the JFFS part
/jffs/$JFFS_SCRIPT & >> $SMBLOG 2>&1 /jffs/$JFFS_SCRIPT & >> $SMBLOG 2>&1
-4) create a script called "startup.jffs.sh" on the share:<br>+4. create a script called "startup.jffs.sh" on the share:<br>
#!/bin/sh #!/bin/sh
Line 179: Line 185:
#or other commands from /jffs #or other commands from /jffs
-5) in the webinterface, enable [[Samba]] with appropriate share/user/pass settings from step 2. Enter "startup.smb.sh" as the script<br>+5. in the webinterface, enable [[Samba]] with appropriate share/user/pass settings from step 2.<br>
-6) reboot<br>+* Enter "startup.smb.sh" as the script<br>
 +6. reboot<br>
<br> <br>
Now you should have the share configured in the web interface mounted at /jffs and should also be able to run 'ipkg' ! Now you should have the share configured in the web interface mounted at /jffs and should also be able to run 'ipkg' !
Line 186: Line 193:
*NOTE: If you are a Linux newbie, and you have problems with file permissions, look at [http://wrt-wiki.bsr-clan.de/index.php?title=Jffs_sharing_in_a_Linux_server this ] *NOTE: If you are a Linux newbie, and you have problems with file permissions, look at [http://wrt-wiki.bsr-clan.de/index.php?title=Jffs_sharing_in_a_Linux_server this ]
-*BONUS NOTE OF HAPPINESS: You need to save the script files in UNIX format, not DOS. One text editor that can do this is UltraEdit. Or Crimson Editor, which is free :o)+*BONUS NOTE OF HAPPINESS: You need to save the script files in UNIX format, not DOS. Windows editors that can do this are UltraEdit. EditPadPro and Crimson Editor, are free.
-===(Depricated) Use Samba Startup Script: Example 3 ===+===Use Startup Script: Example 3 ===
This method requires 2 shares available for DD-WRT, but allows you to have 2 different areas of storage for the router. Some might find it useful. It is an amalgamation of the previous 2 methods. This method requires 2 shares available for DD-WRT, but allows you to have 2 different areas of storage for the router. Some might find it useful. It is an amalgamation of the previous 2 methods.
Line 197: Line 204:
Procedure: Procedure:
-# Enable JFFS<br />+1. Enable [[Samba]]<br />
-# Enable [[Samba]]<br />+2. Create a shell script in the Samba share, as follows:<br />
-# Create a shell script in the Samba share, as follows:<br />+
#!/bin/sh #!/bin/sh
 + nvram set sys_enable_jffs2=1 # fake-enable JFFS
umount /jffs umount /jffs
smbmount //<server's IP>/<jffs share name> /jffs -o username=<your username>,password=<your password> smbmount //<server's IP>/<jffs share name> /jffs -o username=<your username>,password=<your password>
-# Enter the name of the script in the Samba startup script box in the GUI<br />+3. Enter the name of the script in the Samba startup script box in the GUI<br />
-# Reboot+4. Reboot
As an example, my DD-WRT has these shares: As an example, my DD-WRT has these shares:

Revision as of 09:58, 22 May 2008

Contents

Introduction

Journalling Flash File System (JFFS/JFFS2) is a re-writable area within a DD-WRT-enabled device.

  • Located in /jffs.

Contents stored:

  • On DD-WRT Device main flash directly on the device (/jffs), or
  • mount --bind /storagelocation /jffs.
    Where storagelocation is
    • Off-board supplemental flash, see SD/MMC_mod, (/mmc/jffs).
    • Network Area Storage [NAS], or Server. (//servername/sharename)
    • JFFS reports as it's own free space, the free space on Share or SD Card, in the Web Interface

Examples of the kind of programs and data are:

  • Custom configuration files
  • NoCat splash pages and configuration
  • ipkg
  • Web-site hosting
  • Many other things.
  • JFFS must be Enabled to use /jffs-dependent applications properly.
    • ipkg requires JFFS enabled; creating a mount-point at /jffs is not enough.

Enable JFFS: Create /jffs

Prerequisites

On 4MB routers, enabling JFFS is made possible by using the DD-WRT Standard-NoKaid builds. NoKaid removes a specific XBOX module, leaving room for JFFS. The Mini builds of DD-WRT also work. This leaves between about 320KB - 700KB+ of available re-writable space in /jffs.

Very old versions of some Linksys routers (no longer manufactured for many years) contained 8MB of Flash and 32MB of RAM, many DD-WRT-capable devices today contain 4MB of Flash memory; a few models from certain manufacturers contain more flash (La Fonera, ASUS WL-500GP), some contain less (2MB). See Supported Devices.

Flash Wear: A Warning

Take care not to store frequently changing files (i.e. often-changing log files) on Flash as this can cause flash wear-out over time.

  • A journalling flash file system is specifically designed to minimize frequent overwrites. * Files which are read many times but written infrequently work well on main flash.
  • Using the space provided by SD/MMC_mod removes wear from occurring on unreplaceable main flash.

Directions for (normal) users: Web Interface

The steps to enable JFFS through the router web page are very specific. To avoid having to reset and reprogram your router, it's smart to make a backup here of your settings. If you follow these steps exactly, it should not lock up.

  1. On the router web page click on Administration.
  2. Scroll down until you see JFFS2 Support.
  3. Click Enable JFFS.
  4. Click Save.
  5. Wait couple seconds, then click Apply.
  6. Wait again. Go back to the Enable JFFS option, click Clean JFFS.
  7. Do not click "Save". Click Apply instead.

The router formats the available space, which is now shown in the JFFS2 Support box.

Directions for (freaky) Linux Gurus: CLI

Enter the following commands from the CLI:

nvram set jffs_mounted=1
nvram set enable_jffs2=1
nvram set sys_enable_jffs2=1
nvram set clean_jffs2=1
nvram set sys_clean_jffs2=1
nvram commit
reboot

Note: Device might reboot/hang after cleaning which might take long; be patient.

WARNING

A user comments: I tried the previous commands under La Fonera firmware v24 RC4 and corrupted all my settings. I had to recover my unit with the procedures on La Fonera flashing page. I am not sure if this is a bug or these commands are not suited for La Fonera. Please try to use the web interface above. It works for me.

To unmount the JFFS:

nvram set sys_enable_jffs2=0
nvram set sys_clean_jffs2=0
nvram set jffs_mounted=0
nvram commit
reboot

Afterwards you can delete these variables with "nvram unset <variable>".

Increasing JFFS Space: Options

Add a 1GB flash memory card

  • Add the SD/MMC mod
    • up to (at least) 1GB storage using common swappable SD memory cards (built-in support with DD-WRT v2.4 and above).
    • Windows XP: WinSCP can be used to transfer files to and from the router, eliminating the need to pull the card out and put it in a card reader on a PC.
    • Linux, Mac OS X: Other SCP transfer applications exist.
    • Removed cards can be accessed natively in Linux or OS X, or on Windows with the Ext2FS driver [[1]].
    • On older firmware (v2.3), the limited JFFS space can be used to hold the drivers to load support for the SD card.

Add USB Storage

  • Routers with USB storage: A Linksys WRTSL54GS or ASUS WL-500g Premium device has USB capability (and more Flash and RAM).
    • Some firmwares (older): Drivers and startup files are stored in a small amount of JFFS space
    • Current firmware: USB support available within current v2.4 builds.

Using CIFS (Samba): Storing /jffs content remotely

One option is to create a share on the network and mount it on the router at /jffs using CIFS. Exact steps at CIFS

Overview:

  • Create a share on a CIFS-capable server called jffs.
    • Standard Microsoft Windows Shared Folder
    • Other operating systems may refer to SMB or Samba
  • In the Web GUI mount the CIFS share at /jffs.
  • Folder /jffs appears locally, while
  • Content actually saved remotely, on
    • NAS (Network Attached Storage)
    • Server (PC or some type of file server)

Costs:

  •  ! Setting up a NAS or Server only for router maybe not economical.
    • Equipment Costs: Requires existing NAS or PC Server
    • Electricity Costs: if entire Server is only serving router
  • Availability Demands: CIFS share must be available
    • 24/7, or whenever the DD-WRT Device is on, and
    • Storing JFFS Share on quality equipment is required (i.e. Raid, Power Backup)
      •  ! Remote storage down means /jffs down, could mean DD-WRT Device down (increases the weak points).
  • Security Limitations: CIFS is limited to Microsoft's SMB protocol, offering up to only 128-bit security.
    • Poor Microsoft Protocol Security is not a problem if the LAN is physically and electronically protected from outside or inside intrusion (i.e. Guards, Guns, Locks, Dogs, Mines, Cameras, Firewalls)
  • LAN Connection Required: NAS cannot be wirelessly attached to DD-WRT Device.
  • Hassle: Using Microsoft Windows as the jffs Server
    •  ! Symbolic links in /jffs may not work.
    •  ! Rename (mv) commands on the /jffs mount may not work.

Most people only own an (inexpensive) Windows PC, and it is ill-advised to use a common Desktop PC as a 24/7 Server.

External Links:
See http://wrt-wiki.bsr-clan.de/index.php?title=Jffs.
Linux Newbies: http://wrt-wiki.bsr-clan.de/index.php?title=Jffs_sharing_in_a_Linux_server.

The following non-updated (as of 20080520) examples may be out of date:

  • CIFS was broken on many builds this update author tried. Although other people have reported it working - It may be a limitation of BusyBox CIFS when using NTLMv2 and 128-bit encryption.
  • (Mostly) only structural and spelling mistakes were corrected below (20080522)

Use a Startup Script: Example 1

  1. Read how to use Startup Scripts. Then place the following into your startup script to mount the JFFS partition:
umount /jffs #unmounts existing /jffs
mount.cifs //<server's IP>/jffs /jffs -o username=<the username>,password=<the password>
nvram set sys_enable_jffs2=1 # fake-enable JFFS. Enable but do not commit (no format).
  • NOTE: username is Guest when hosted on Windows XP & 'Simple File Sharing' is enabled - This is the default on Windows XP Home.

Use Startup Scripts: Example 2

This method uses 2 scripts: The first will be started from /tmp/smbshare & mount the samba share under /jffs and start the other script. The second will be started from /jffs and unmount /tmp/sambashare.

Instructions:

  1. JFFS is 'fake-enabled' in the script.
    • With the JFFS NVRAM variable set to true (but not committed!), and the /jffs folder present, ipkg works.
  2. Enable CIFS (see also Jffs_sharing_in_a_Linux_server).
  3. Create a script called "startup.smb.sh" on the share:
#!/bin/sh

#smb settings: same as the settings in the webgui
# change these to mount a different share than the current at /jffs
SMBSHARE=`nvram get samba_share`
SMBUSER=`nvram get samba_user`
SMBPASS=`nvram get samba_password`

#script to start in /jffs once we're done mounting
JFFS_SCRIPT=startup.jffs.sh

#where to log what is happening
export SMBLOG=/tmp/smblog
 
####### ok, here we go... ;) #######
echo "" > $SMBLOG
cd /

#make sure JFFS is unmounted.
echo "umounting /jffs if nescessary..." >> $SMBLOG
umount /jffs >> $SMBLOG 2>&1

#mount our smbshare at /jffs
echo "mounting $SMBSHARE at /jffs..." >> $SMBLOG
#Depricated: smbmount $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1
mount.cifs $SMBSHARE /jffs -o username=$SMBUSER,password=$SMBPASS >> $SMBLOG 2>&1

#hand over startup to the JFFS part
/jffs/$JFFS_SCRIPT & >> $SMBLOG 2>&1

4. create a script called "startup.jffs.sh" on the share:

#!/bin/sh

#wait a sec, just to be sure the smb startup script has quit
sleep 1

#unmount the smbshare, its identical to /jffs now anyway :)
umount /tmp/smbshare

#to enable ipkg, we fake JFFS enabled.
#since theres no commit, this wont actually enable JFFS
nvram set sys_enable_jffs2=1

#ready to do specific stuff, like starting services
#or other commands from /jffs

5. in the webinterface, enable Samba with appropriate share/user/pass settings from step 2.

  • Enter "startup.smb.sh" as the script

6. reboot

Now you should have the share configured in the web interface mounted at /jffs and should also be able to run 'ipkg' !

  • NOTE: If you are a Linux newbie, and you have problems with file permissions, look at this
  • BONUS NOTE OF HAPPINESS: You need to save the script files in UNIX format, not DOS. Windows editors that can do this are UltraEdit. EditPadPro and Crimson Editor, are free.

Use Startup Script: Example 3

This method requires 2 shares available for DD-WRT, but allows you to have 2 different areas of storage for the router. Some might find it useful. It is an amalgamation of the previous 2 methods.

Things you need:

  • 2 writeable shares on another computer/server

Procedure:

1. Enable Samba
2. Create a shell script in the Samba share, as follows:

#!/bin/sh
nvram set sys_enable_jffs2=1 # fake-enable JFFS
umount /jffs
smbmount //<server's IP>/<jffs share name> /jffs -o username=<your username>,password=<your password>

3. Enter the name of the script in the Samba startup script box in the GUI
4. Reboot

As an example, my DD-WRT has these shares:

  • /tmp/smbshare located in a shared folder on my NAS called ddwrt, which contains scripts (including startup.sh containing the above commands), yet to be installed packages and other static data
  • /jffs located in a shared folder called jffs, which contains program data for installed packages

Testing

Log in to the shell (Telnet or SSH) and run:

mkdir -p /jffs/tmp/ipkg
ipkg update
ipkg list