Monday, March 21, 2011

ORA-27054: NFS file system where the file is created or resides is not mounted with correct options

All I seem to post are my issues...well, technical issues.

On Friday I encountered ORA-08103: object no longer exists. The result there, I had to re-re-re-re-re-load the files. Once I did that, I realized that the file name had actual meaning...In the table itself was a column called MONTH which was relative to the fiscal year (July - June). Guess where fiscal year was? Yup, the file name. Ugh.

So, as soon as I got the data loaded, I did this:
RMAN>BACKUP DATABASE;
I didn't want to go through the corrupted block issue for a 3rd time. After some more work, moving data around, I checked out the RMAN docs and decided to do perform an incremental backup. Well, I'm still learning, I apparently did the whole thing.
RMAN>BACKUP DATABASE INCREMENTAL LEVEL 0;
OK, now I'm running out of space, I have 500 GB allocated to this VM.

I read through the RMAN docs some more, BACKUP BACKUPSET. Yeah, backing up your backup. Inception.

I wanted to save this to my Shared Folders that now auto-mount (as of VirtualBox 4.0).
RMAN> BACKUP BACKUPSET 4
2> FORMAT '/media/sf_files/oradata/TESTING/20110320_%u'
3> DELETE INPUT;

Starting backup at 20-MAR-11
using channel ORA_DISK_1
channel ORA_DISK_1: input backup set: count=13, stamp=746274912, piece=1
channel ORA_DISK_1: starting piece 1 at 20-MAR-11
channel ORA_DISK_1: backup piece /u01/app/oracle/product/11.2.0/dbhome/dbs/0dm7mg30_1_1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 03/20/2011 21:36:53
ORA-19504: failed to create file "/media/sf_files/oradata/TESTING/20110320_0dm7mg30"
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 3
ORA-19600: input file is backup piece (/u01/app/oracle/product/11.2.0/dbhome/dbs/0dm7mg30_1_1)
ORA-19601: output file is backup piece (/media/sf_files/oradata/TESTING/20110320_0dm7mg30)
Checking the Google Machine took me to Kevin Closson's post Mount Options for Oracle over NFS. It’s All About the Port.. Those options are the following:
rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0
OK, I'm still new to Linux and now my Shared Folders auto-mount, WTF do I do? More Google Machine goodness took me to a post on the OTN Forums with the exact command I needed to run (which is of course, why I am writing this up now...).
[root@medicaid media]# mount -o rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0 -t nfs 
oraclenerd:/media/data_2/files/oradata/TESTING /media/test
See that little trick I picked up? I used oraclenerd:/media/data_2/... instead of relying on the VirtualBox magic. Nice.

No errors, I then checked out the permissions:
[root@medicaid media]# ls -la
total 36
drwxr-xr-x 6 root root 4096 Mar 21 01:37 .
drwxr-xr-x 28 root root 4096 Mar 18 02:13 ..
-rw-r--r-- 1 root root 0 Mar 18 02:13 .hal-mtab
drwxrwx--- 1 root vboxsf 4096 Mar 4 04:18 sf_data
drwxrwx--- 1 root vboxsf 8192 Mar 8 13:59 sf_exp
drwxr-xr-x 1 root vboxsf 4096 Mar 20 21:18 sf_files
drwxr-xr-x 2 root root 4096 Mar 21 01:37 test
OK, let's change those:
[root@medicaid media]# chown oracle:dba test/ -R
[root@medicaid media]# ls -la
total 36
drwxr-xr-x 6 root root 4096 Mar 21 01:37 .
drwxr-xr-x 28 root root 4096 Mar 18 02:13 ..
-rw-r--r-- 1 root root 0 Mar 18 02:13 .hal-mtab
drwxrwx--- 1 root vboxsf 4096 Mar 4 04:18 sf_data
drwxrwx--- 1 root vboxsf 8192 Mar 8 13:59 sf_exp
drwxr-xr-x 1 root vboxsf 4096 Mar 20 21:18 sf_files
drwxr-xr-x 2 oracle dba 4096 Mar 21 01:37 test
Sweet. Now, run the RMAN backup of a backup again:
RMAN> BACKUP BACKUPSET 4
2> FORMAT '/media/test/2011_%u'
3> DELETE INPUT;

Starting backup at 21-MAR-11
using channel ORA_DISK_1
channel ORA_DISK_1: input backup set: count=13, stamp=746274912, piece=1
channel ORA_DISK_1: starting piece 1 at 21-MAR-11
channel ORA_DISK_1: backup piece /u01/app/oracle/product/11.2.0/dbhome/dbs/0dm7mg30_1_1
No errors...and it seems to be running fine. If it breaks, I'll update.

Update: 03/21/2011 02:41 AM
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 03/21/2011 02:39:39
ORA-19502: write error on file "/media/test/2011_0dm7mg30", block number 10173824 (block size=8192)
ORA-27072: File I/O error
Linux-x86_64 Error: 11: Resource temporarily unavailable
Additional information: 4
Additional information: 10173824
Additional information: 630784
ORA-19502: write error on file "/media/test/2011_0dm7mg30", block number 10173824 (block size=8192)
Barnacles.

4 comments:

jpiwowar said...

If you want to save some of that 500GB, might I suggest "backup as compressed backupset"? :)

jpiwowar said...

Oh heck, nevermind, totally misread the early parts of your post. I'll try to keep up next time. :-P

oraclenerd said...

@jp

You may suggest that. I haven't been through the entire doc just yet...bonus points for the table data already being compressed though?

Tom said...

Maybe a VBox bug? Try to create an NFS mount that bypasses the shared_folder stack. Just straight up NFS.