Search This Blog

Thursday, January 13, 2011

SMB Streaming issues in Dharma - Using NFS for better streaming

One thing I have noticed about Dharma (and I presume it's actually more to do with the underlying lucid lynx than Dharma per se, althought I don't know) - is MUCH greater flakiness with low quality networks, like wireless and Ethernet over Power.

My Camelot set up happily streamed all 720p and almost all 1080p without glitches using Ehtnet over Power (EoP). On the same hardware, same network, Dharma can barely manage 720p, with regular buffering issues. 1080p - fuggedaboutit!

There are quite a few reports of this, not a lot of solutions to date though. I suspect it's to do with default SMB mounting options and an overly small buffer or something, but in general SMB is not known to be a wonderfully efficient protocol. It also means tht you need to use Windows name resolving which can be painfully slow at times, so all in all, it's not perfect.

The solution seems to be to use NFS rather than SMB to share the files.

With this here I can reliably stream 720p and while 1080p may buffer a bit at the beginning it seems to settle down (I’ve only tried it for 10 mins or so, as the second machine is not really a machine I actually use much, it’s really more for my wife and kids to use during the day!). This is with my power net thingy showing an amber or red signal, not green – so medium to bad connection (40 to 80 mb). In all this is at least as good as with Camelot now I’d say. This was all tested while my wife was watching 720p stuff on the main machine too so the network was being kept busy with that as well.

I think this is the best option but the Openelec people are looking at SMB sharing issues as ewll.

So, to try it isn’t that hard:

Basically, you run a little utility on your Windows server called Hanewin NFS (http://www.hanewin.net/nfs-e.htm)

Install that and then you go into the start menu, run the little administrator proggy
(Hanewin->NFS->NFS Server (right click and choose run as administrator).

On the exports tab, set up your shares:

“Full path on your server” –name:WhateverWithoutSpaces –public

Like so:

"F:\Movie Library 01" -name:MovieLibrary01 -public
"G:\Movie Library 02" -name:MovieLibrary02 -public

"M:\Movie Library 03" -name:MovieLibrary03 -public

"H:\TV Library 03" -name:TVLibrary03 -public

"I:\TV Library 04" -name:TVLibrary04 -public

"J:\TV Library 05" -name:TVLibrary05 -public

K:\ -name:Store06 -public

K:\Thumbnails -name:Thumbnails -public

"K:\Music Library" -name:MusicLibrary –public


Save the file, and restart the server using the button on that tab, and you should see a list of your shares come up if you did it right (under the Directory column).

In the Start->Hanewin->NFS, choose ‘Start NFS Server’ and ‘Start SunRPC Port Daemon’. This gets it actually going. Turn off you Windows Firewall for you internal network. Leave it on for the external network, but there’s no real issue turning it of inside your own network really. You can probably bind some specific ports and open only those if you want, but let's keep this simple for now.

Now, mount them on your Openelec machine using netmount.conf.

nfs | serverIP:/ShareNameNoSpacesAsInHanewinExportsFile |/storage/mount/Whatever You Like

nfs | 192.168.1.51:/Thumbnails | /storage/.xbmc/userdata/Thumbnails
nfs | 192.168.1.51:/MovieLibrary01 | /storage/mount/Movie Library 01

nfs | 192.168.1.51:/MovieLibrary02 | /storage/mount/Movie Library 02

nfs | 192.168.1.51:/MovieLibrary03 | /storage/mount/Movie Library 03

nfs | 192.168.1.51:/TVLibrary03 | /storage/mount/TV Library 03

nfs | 192.168.1.51:/TVLibrary04 | /storage/mount/TV Library 04

nfs | 192.168.1.51:/TVLibrary05 | /storage/mount/TV Library 05

nfs | 192.168.1.51:/Store06 | /storage/mount/Store 06

nfs | 192.168.1.51:/MusicLibrary | /storage/mount/Music Library


Reboot your machine and the shares should come up. Everything should be much snappier and more reliable if my network here is any guide.

(Note I have got my .xbmc/userdata/Thumbnails folder on a network share now – so with the MySQL db, watched status/thumbnails etc, are all brought in from the server and will survive upgrades etc.)

The NFS shares seem to connect more quickly at start up too, so so far I haven’t found any issues at all.

I'm now using NFS on both our boxes and so far it's noticeably more responsive (particularly with things like looking at photos), and much reduced buffering with PoE to our second machine (our main machine is using ethernet so this wasn't an issue).

13 comments:

  1. This is good to know. I've had buffering issues streaming from my WHS when copying large files to it. Turns out there is a lot of controversy regarding a service that MS is running on WHS. If I disable the Drive Extender Migrator Service from running, I have no buffering issues. As a result, I've scheduled the service to only run when we're not usually in the house.

    Thanks for the info. If I have any more problems, I may look into doing this.

    ReplyDelete
  2. Good writeup, thanks for posting. I've got a similar solution running with a shared thumbnails directory shared among all servers and shared library among all servers using the MySQL DB in advancedsettings. One limitation (AFAIK) with using a shared DB between Windows & Linux is that you are stuck using SMB on all machines so that the paths on all machines are the same and the library can be shared among all servers.

    Is there a way that a Windows machine running XBMC could access the movie directories via CIFS as well so that all machines share the same paths to the content, or would this only work for Linux machines accessing a Windows file share?

    ReplyDelete
  3. I just built an OpenELEC Beta 4/Shuttle XS35GT combination, using a OCZ Vertex 2 (60GB) inside the XS35GT, and a wired ethernet connection to my Netgear router. I also have a Netgear ReadyNAS Duo (initially configured with SMB only), and have been VERY disappointed with the streaming performance - excessive pause for buffering, large initial wait to begin video).

    I went to openELEC with the hope that the same issue I was experienced with a Dharma install would be alleviated - but it wasn't! Should I blame SMB? I've enabled NAS on the ReadyNAS in hope that my dreams of streaming media from a redundant repository on the NAS to my XBMC wasn't a pipedream?

    Comments/suggestions on tuning?

    ReplyDelete
  4. My system is a Windows 7 Ultimate server, sharing files via Hanewin NFS to two linux clients (Openelec). But the basica principle is the same - you use any one of the possible systems (i.e. openelec's .config/netmount.conf, or on a Live/XBMCFreak install you use either fstab or autofs to mount your NFS shares) - if using multiple machines & MySQL (as I am), just specify the mountpoint to be the same on all machines (I use /storage/mount/Share 01 etc). Thus the local paths are all the same, and the MySQL DB sees the references as all the same. Works a treat.

    Now, how to do the same on Windows client so they all refer to the same spot? I am not immediately sure but I suspect you could use an NTFS junction point or something. And I know you can mount NFS shares in Windows too, so it should be possible. I'd imaging a clever clogs in the forum could help with this.

    I actually got info straight from one of the XBMC devs that with Dharma they moved from XBMC internal SMB code to system SMB code. Hence the sudden occurence of buffering issues, but I presume it means a lot less to maintain for them. In any case, NFS works WAY better - obviously MUCH MUCH better with dodgy networks, but even over full wired ethernet it makes things that bit snappier and more responsive. Definitely worth the hassle.

    (I would think a ReadyNAS can share with NFS too, they used to before Netgear gobbled them up, anyway).

    ReplyDelete
  5. Quote:
    -----
    Now, how to do the same on Windows client so they all refer to the same spot? I am not immediately sure but I suspect you could use an NTFS junction point or something. And I know you can mount NFS shares in Windows too, so it should be possible. I'd imaging a clever clogs in the forum could help with this.
    -----

    For anyone who lands here in the future trying to accomplish this, I thought I'd post back my results. I spent several hours researching this and it's not possible by any means I can figure out (though I will be happy to be proven wrong).

    The problem is that for the video paths to work the same on all systems, you either

    - have to use a URL protocol (i.e. SMB, which we rule out for reasons above),

    - force Linux to recognize a Windows drive letter format (C:\), which Windows will not do.

    - force Windows to recognize a linux root-like file structure (/mnt), which it throws up all over.

    I tried several other more creative options (smylinks, junctions, directly accessing \\ directories, etc.) and in the end, I succumbed to the fact that this isn't possible.

    Given that I have more Linux systems than Windows, I'll probably just leave my Windows systems to fend for themselves DB-wise and get all of the Linux boxes reading from the same DB and call it a small victory.

    ReplyDelete
  6. Did you ask on the forum (try the MySQL thread) - I'm sure someone has done this (although maybe not with NFS I guess). Technically NFS does have a URL protocol of:

    nfs://:

    (see http://www.faqs.org/rfcs/rfc2224.html)

    ..but I am not sure XBMC supports this - have you tried the above exactly?

    ReplyDelete
  7. nfs://<host>:<port><url-path>

    Ok that URL got munged...

    ReplyDelete
  8. Thanks for the comment. I'll throw the question out in the XBMC thread tomorrow to see if anyone has figured it out. Re: the nfs://, yep, that was one of my high hopes when I started, but xbmc doesn't support that protocol (yet!).

    ReplyDelete
  9. Yeah ok I suspect you're stuffed then. Really paths shouldn't be part of a good DB anyway, if you ask me. And what with my status as a never-developed-a-single-line-of-xbmc-code my vote counts for something, right :)

    ReplyDelete
  10. For anyone curious, I did some testing in the last week of NFS vs. SMB performance and posted the results on my blog. Bossanova - thanks for the original post as it was definitely what gave me the idea to test the differences for XBMC streaming, and they are huge!

    Test results:
    http://www.technazgul.com/2011/02/nfs-vs-smb-throughput-for-hd-media.html

    ReplyDelete
  11. It's an old trick from my popcorn hour days (well technically hdx 1000). NFS works much better with them as well.

    I really see no disadvantages and a lot of the wierd pauses when whipping between folders seem to have been SMB caused. The whole thing is just that bit mroe responsive and glitch free now.

    ReplyDelete
  12. I've the unresponsible hang issue with xbmc which I want to solve using NFS shares. I've set up the shares in Hanewin and started ‘Start NFS Server’ and ‘Start SunRPC Port Daemon’ but how will this be added to xbmc ? It still using my SMB shares.
    Btw I don't have openelec just a normal XBMC Live Dharna 10.1 installation.

    ReplyDelete
  13. You'll need to use autofs (or fstab) to mount the shares outside of XBMC - see:

    http://xbmcstuff.bossanova808.net/2010/09/solution-to-xbmc-idle-hanging-problem.html

    ReplyDelete