Topic: 2 gig avi file

[SalaTar]

Question:
I have 2 2.8 Gig avi files I need to move to my array (raid 5 ext2 )
I have renamed them but still cant get them to go to server without a error.
Could someone point me to thread all I find are backup threads about 2 gig limit.

[Kelvin]

SME versions up to 5.5 cannot store any single file larger than 2GB (has nothing to do with the name).

Kelvin

[Nathan Fowler]

You could split the file into multiple volumes using a number of compression programs (Zip, RAR, Gzip, etc).  The issue is that ext2 cannot support a single file greater that 2GB in size (I believe this is due to a 32bit filesystem limitation).  SME 5.6 uses ext3 (from what I understand).

Nathan

[Kelvin]

With SME up to 5.5, the 2GB limit also applies to smbfs, samba, etc,so it's not only a file system issue.

However, when researching the smbfs issue, I came across a number of posts which suggests that ext2 is not what's causing the 2GB limit and in fact ext2 can support files larger than 2GB. It's the other bits (like the smbfs and others) that are the limiting factors. Unfortunately, as I was actually researching smbfs and not ext2, I did not pursue the matter any deeper.

Kelvin

[SalaTar]

Seems its Samba that does this.
Its only 2 files on or first SME box in a novell network (of 100's servers...take note Mitel)
Looking into upgrading kernel and samba..then fixing ext2 fs or going ext3 on that array (its just home/e-smith/files)

"The Ext2fs supports standard Unix file types: regular files, directories, device special files and symbolic links.
Ext2fs is able to manage filesystems created on really big partitions. While the original kernel code restricted the maximal filesystem size to 2 GB, recent work in the VFS layer have raised this limit to 4 TB. Thus, it is now possible to use big disks without the need of creating many partitions.
Ext2fs provides long file names. It uses variable length directory entries. The maximal file name size is 255 characters. This limit could be extended to 1012 if needed.
Ext2fs reserves some blocks for the super user (root). Normally, 5% of the blocks are reserved. This allows the administrator to recover easily from situations where user processes fill up filesystems. "

[Nathan Fowler]

SalaTar, bypass Samba and use SCP/SFTP/FTP, I think you're may see that ext2 is a limiting factor at 2GB, not samba.  If I'm wrong, please let me know and I'll make quite a few people running SME on ext2 rest easier, including myself.

Thanks,
Nathan

[Des Dougan]

ext2 is not the limiting factor - it's the 2.2 kernel. 2.4 removed the restriction. ext3 is an extension to ext2 - simply adding journalling - which is why (AFAIK) we are able to upgrade from 5.x to 5.6 without reformatting disks.

Des Dougan

[Nathan Fowler]

Thanks Des, this is good information to know.  Now, is it possible to recompile (or patch) a 2.2.x series kernel to increase the file size limitation, or is this something that isn't patchable in 2.2.x and is only available in 2.4.x?

[Kelvin]

Hi SalaTar / Nathan,

Have a look here :-

http://marc.theaimsgroup.com/?l=samba&m=104377567606842&w=2

and

http://www.jetico.com/linux/bcrypt-help/l_requi.htm


Kelvin

[Kelvin]

Also :-

http://www.suse.de/~aj/linux_lfs.html

By the way, just for the record, largest single file size on FAT32 is 4GB and not 2GB as suggested by the 2nd article of my previous post.

Kelvin

[SalaTar]

http://www.namesys.com/install.html

[Nathan Fowler]

SalaTar, bypass Samba and see what limitation you see, I want to see if the 2GB
limitation exists at the kernel/fs level, or if it's a samba issue....

Kelvin/Salatar, thanks for the links, I'll check it out.

[Kelvin]

Nathan,

The problem is not at samba level. Remember the problem with large backups to desktop and why they cannot be restored ? That does not use samba or smbfs but a http protocol, I believe. The reason you cannot restore the large backup is because the temp file created by the server cannot exceed 2GB.

In any case, it's easy enough to test. At the server prompt, just cat together a number of large files (CD-ROM iso files are great for this purpose). Once you get to 2GB you should get an error.

Kelvin