Home » RDBMS Server » Backup & Recovery » Archivelog Backup in 2 Node RAC
Archivelog Backup in 2 Node RAC [message #275374] Fri, 19 October 2007 12:37 Go to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Mad
Hello good friends,
I am having a problem backing up my archivelogs. I have research everything posting here but non seems to help. I have try backing up using RMAN & Grid Control but still end up with the same issue. I am able to do a full back but not the archivelog. My archive destinations are:
Node 1 - /u02/oracle/admin/prod/arch/prod1
Node 2 - /u02/oracle/admin/prod/arch/prod2

Terminal window – If I cd into prod2 from node 1 I don’t see archive log listed and vise-versa. Everytime I attempt to backup from a node—I always receive the error message: expected archived log not found at the other node archivelog directory. For example:

RMAN> connected to target database: PROD (DBID=35109652)
using target database controlfile instead of recovery catalog
RMAN> echo set on
RMAN> set command id to 'BACKUP_PROD.WORLD__101907094924';
executing command: SET COMMAND ID
RMAN> backup device type disk tag 'BACKUP_PROD.WORLD__101907094924' archivelog all not backed up;
Starting backup at 19-OCT-07
current log archived
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=1496 devtype=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: sid=1413 devtype=DISK
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 10/19/2007 09:50:21
RMAN-06059: expected archived log not found, lost of archived log compromises recoverability
ORA-19625: error identifying file /u02/oracle/admin/prod/arch/prod2/arch_0002_0000000122_623260315.arc
ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory
Additional information: 3

RMAN> delete noprompt archivelog until time 'sysdate - 7' backed up 1 times to device type disk;
released channel: ORA_DISK_1
released channel: ORA_DISK_2
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=1496 devtype=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: sid=1413 devtype=DISK
RMAN> exit;

I am thinking that the node are not seeing the archivelogs for each other. My specifications are:

AIX 5
2 nodes RAC
Raw device
Let me know if you need any other info to help.

If there is an expert that willing to spend time on the phone with me tonight I will also appreciate that.

Godd bless

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275387 is a reply to message #275374] Fri, 19 October 2007 13:37 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
How are you allocating your channels? You should allocate your channels as such:

ALLOCATE CHANNEL c1 DEVICE TYPE DISK CONNECT 'SYS/oracle@prod1';
ALLOCATE CHANNEL c2 DEVICE TYPE DISK CONNECT 'SYS/oracle@prod2';

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275720 is a reply to message #275387] Mon, 22 October 2007 08:59 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Hello and thanks for the response. Below is the error I got after allocating channels as you suggested.


oracle:@cernappdbs1:/backup/prod> export ORACLE_SID=prod1
oracle:@cernappdbs1:/backup/prod> rman target / catalog rman/xxxx@rmandb

Recovery Manager: Release 10.1.0.5.0 - 64bit Production
Copyright (c) 1995, 2004, Oracle. All rights reserved.
connected to target database: PROD (DBID=35109652)
connected to recovery catalog database

RMAN> RUN
2> {ALLOCATE CHANNEL c2 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'rman/xxx@prod1';
3> ALLOCATE CHANNEL c1 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'rman/xxx@prod2';
4> backup (archivelog all);
5> }

allocated channel: c2
channel c2: sid=1459 devtype=DISK

released channel: c2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-12001: could not open channel c1
RMAN-10008: could not create channel context
RMAN-10003: unable to connect to target database
ORA-01017: invalid username/password; logon denied

Channel c2 was allocated because I am running the backup from that node. It seems RMAN is unable to allocate channel to node 2. Do you know what the problem is? My back up directory is “/backup/prod/backup_db” and is visible on node 1 only. I don’t think this make a different since the archivelogs are known by both nodes. I will like to add that I have double check and the issue does not have anything to do with wrong password. I changed the password several times--commit and retry but now way.

[Updated on: Mon, 22 October 2007 09:22]

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275793 is a reply to message #275720] Mon, 22 October 2007 13:18 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
You need to connect as SYS in the allocation of your channel.

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275796 is a reply to message #275793] Mon, 22 October 2007 13:35 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Good afternoon my friend,
I connected as sys but still had the same problem:

RMAN> RUN
2> {ALLOCATE CHANNEL c2 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SYS/change_on_install@prod1';
3> ALLOCATE CHANNEL c1 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SY/change_on_install@prod2';
4> backup (archivelog all);
5> }

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-12001: could not open channel c2
RMAN-10008: could not create channel context
RMAN-10003: unable to connect to target database
ORA-01017: invalid username/password; logon denied

I know the password is correct because I try it several times...lock key is off. Here is my RMAN perm configuration----which should have no effect on channel allocation if I am right:

RMAN configuration parameters are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1;
CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE DEFAULT DEVICE TYPE TO DISK;
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '/u01/oracle/product/10.1.0.5/db/dbs/backup_ctr_spfile/%F';
CONFIGURE DEVICE TYPE DISK PARALLELISM 2 BACKUP TYPE TO BACKUPSET;
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE CHANNEL 1 DEVICE TYPE DISK CONNECT 'sys/change_on_install';
CONFIGURE CHANNEL 2 DEVICE TYPE DISK CONNECT 'sys/change_on_install';
CONFIGURE CHANNEL DEVICE TYPE DISK CONNECT '*';
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/oracle/product/10.1.0.5/db/dbs/snapshot_ctr_file/scontrolfile.ctl';

Thanks for your help so far...

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275820 is a reply to message #275796] Mon, 22 October 2007 17:38 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
As a test, try to clear your channels with:

CONFIGURE CHANNEL 1 DEVICE TYPE DISK CLEAR;
CONFIGURE CHANNEL 2 DEVICE TYPE DISK CLEAR;
CONFIGURE CHANNEL DEVICE TYPE DISK CLEAR;


Can connect remote as SYSDBA with simply SQL*Plus?

Also, I assume that's a typo for the channel 1 allocation for the SYS name?

youngb912 wrote on Mon, 22 October 2007 14:35


RMAN> RUN
2> {ALLOCATE CHANNEL c2 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SYS/change_on_install@prod1';
3> ALLOCATE CHANNEL c1 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SY/change_on_install@prod2';
4> backup (archivelog all);
5> }

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275825 is a reply to message #275820] Mon, 22 October 2007 20:04 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Hi, thanks again for the help. I cleared the various channel configurations, and try the backup but it didn’t work. The channels couldn’t be allocated. I then try another means---with my last attempt the channels were allocated but it claimed not to find the archivelogs. I have done CROSSCHECK several times so I didn’t do it this time. Here are the outputs. It shows that I can connect to SQL*PLUS as sysdba, and also the archive log:

oracle:@ppdbs1:/home/oracle> export ORACLE_SID=prod1
oracle:@ppdbs1:/home/oracle> rman target / catalog rman/rman@rmandb

connected to target database: PROD (DBID=35109652)
connected to recovery catalog database

RMAN> RUN
2> {ALLOCATE CHANNEL c2 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SYS/change_on_install@prod1';
3> ALLOCATE CHANNEL c1 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'SY/change_on_install@prod2';
4> backup (archivelog all);
5> }

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-12001: could not open channel c2
RMAN-10008: could not create channel context
RMAN-10003: unable to connect to target database
ORA-01017: invalid username/password; logon denied

The try below is different but show some advancement – maybe this can lead to somthing

RMAN> RUN
2> {ALLOCATE CHANNEL C1 DEVICE TYPE DISK;
3> ALLOCATE CHANNEL C2 DEVICE TYPE DISK;
4> BACKUP ARCHIVELOG ALL;
5> }

allocated channel: C1
channel C1: sid=1460 devtype=DISK

allocated channel: C2
channel C2: sid=1367 devtype=DISK

Starting backup at 22-OCT-07
current log archived
released channel: C1
released channel: C2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 10/22/2007 20:45:13
RMAN-06059: expected archived log not found, lost of archived log compromises recoverability
ORA-19625: error identifying file /u02/oracle/admin/prod/arch/prod2/arch_0002_0000000186_623260315.arc
ORA-27037: unable to obtain file status
IBM AIX RISC System/6000 Error: 2: No such file or directory
Additional information: 3


RMAN> exit
Recovery Manager complete.
oracle:@ppdbs1:/home/oracle> export ORACLE_SID=prod1
oracle:@ppdbs1:/home/oracle> sqlplus /nolog

SQL*Plus: Release 10.1.0.5.0 - Production on Mon Oct 22 20:51:34 2007

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> connect sys/ as sysdba
Connected.
SQL>

Here is the archivelog directory. I’m login from prod1/node1. I am able to see the archive logs for prod1 but not prod2. When I login to prod2/node2 I can see its archive logs but not that for node1. It seems they are not visible to each other. How can I resolve this? Your help will be highly appreciated.

oracle:@cernappdbs1:/u02/oracle/admin/prod/arch> ls
prod1 prod2
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch> cd prod1
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch/prod1> ls
arch_0001_0000000699_623260315.arc arch_0001_0000000721_623260315.arc
arch_0001_0000000700_623260315.arc arch_0001_0000000722_623260315.arc
and many more for node1
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch/prod1> cd ..
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch> ls
prod1 prod2
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch> cd prod2
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch/prod2> ls
oracle:@cernappdbs1:/u02/oracle/admin/prod/arch/prod2>

As you can see there is none visible on prod2 while login through node1. If I where login to node2 it will be visible but not that for node1. Pls let me know if I'm posting too much info.

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #275826 is a reply to message #275825] Mon, 22 October 2007 20:16 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
youngb912 wrote on Mon, 22 October 2007 21:04

... It shows that I can connect to SQL*PLUS as sysdba...
oracle:@ppdbs1:/home/oracle> export ORACLE_SID=prod1
oracle:@ppdbs1:/home/oracle> sqlplus /nolog

SQL*Plus: Release 10.1.0.5.0 - Production on Mon Oct 22 20:51:34 2007

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> connect sys/ as sysdba
Connected.
SQL>

Your test connection as SYS was only to the local database (prod1). You need to perform the SQL*Plus connection as SYS from prod1 node as follows:
sqlplus sys/password@prod2 as sysdba


After I had a closer at your code, also try a local connection to your database as:
sqlplus sys/password@prod1 as sysdba


In addition, check to ensure that your REMOTE_LOGIN_PASSWORDFILE = EXCLUSIVE for each instance. Your password file may not be available or incorrect.

[Updated on: Mon, 22 October 2007 20:36]

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276008 is a reply to message #275826] Tue, 23 October 2007 11:28 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Hi, thanks for your help again. You were right!!! The REMOTE_LOGIN_PASSWORDFILE was changed but sys password was not changed. I made the changes. Initially I couldn't connect from node 1 --sqlplus sys/password@prod2 as sysdba. That is now resolved. I ran the below command from NODE1, (which has the backup directory -/backup) and the archivelog backup proceeded without the numerous error I was having.

RMAN> RUN
2> {ALLOCATE CHANNEL c1 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'sys/oracle@prod1';
3> ALLOCATE CHANNEL c2 DEVICE TYPE DISK FORMAT '/backup/prod/backup_db/%U' MAXPIECESIZE 2G CONNECT 'sys/oracle@prod2';
4> backup (archivelog all);
5> }

allocated channel: c1
channel c1: sid=1446 devtype=DISK

allocated channel: c2
channel c2: sid=1584 devtype=DISK

Starting backup at 23-OCT-07
current log archived
channel c1: starting archive log backupset
channel c1: specifying archive log(s) in backup set
input archive log thread=1 sequence=699 recid=843 stamp=636292388
And soon on until....

channel c1: starting piece 1 at 23-OCT-07
channel c1: finished piece 1 at 23-OCT-07
piece handle=/backup/prod/backup_db/3oiv78ms_1_1 comment=NONE
channel c1: starting piece 2 at 23-OCT-07
channel c1: finished piece 2 at 23-OCT-07
piece handle=/backup/prod/backup_db/3oiv78ms_2_1 comment=NONE
channel c1: backup set complete, elapsed time: 00:02:21
released channel: c1
released channel: c2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================

RMAN-03009: failure of backup command on c2 channel at 10/23/2007 11:42:39
ORA-19504: failed to create file "/backup/prod/backup_db/3niv78ea_1_1"
ORA-27040: file create error, unable to create file
IBM AIX RISC System/6000 Error: 2: No such file or directory

The "/backup/prod/backup_db” directory is located on node1. Does this mean node2 is unable to write to that directory? I guess I have to make it visible from node2. I will await your reply before I proceed with such a thing.

Thanks for helping and being patience.

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276024 is a reply to message #276008] Tue, 23 October 2007 12:33 Go to previous messageGo to next message
DreamzZ
Messages: 1666
Registered: May 2007
Location: Dreamzland
Senior Member
Check permissions and double check weather the directory exists?

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276027 is a reply to message #276024] Tue, 23 October 2007 12:40 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
The directory exist on node1. Does it have to be on node2 or need to be visible? I am backing up to the backup directory on node1. What permission you referring to? Here is the direcory as it exist on node1:

drwxr-xr-x 12 oracle dba 4096 Oct 16 14:01 backup

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276028 is a reply to message #276027] Tue, 23 October 2007 12:44 Go to previous messageGo to next message
DreamzZ
Messages: 1666
Registered: May 2007
Location: Dreamzland
Senior Member
Is it on local machine or shared storage?
If not on shared storage then create it on node 2 and then try again.

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276032 is a reply to message #276028] Tue, 23 October 2007 12:59 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
It is on a shared storage - SAN. here are more detail:

backupvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
loglv06 jfs2log 1 1 1 open/syncd N/A
feelv00 jfs2 4264 4264 10 open/syncd /backup

I am wondering how to make this visible to node2...

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276036 is a reply to message #276032] Tue, 23 October 2007 13:14 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
You mentioned above that /backup/prod/backup_db/ exists only on node1 ? Is it local to node1 or is it actually shared ?

If you have shared storage where you can create the directory, that would be your best bet. Is your shared storage RAW ?

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276037 is a reply to message #276036] Tue, 23 October 2007 13:17 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
I said local because the directory is not listed when you cd / in node 2. Unlike node1 (cd /---will show it to you). Ebrian has been very helpful so far. I beleive my current problem might be more of a UNIX, but since I don't have a UNIX admin I will have to work on getting it resolve.



  • Attachment: note.doc
    (Size: 24.50KB, Downloaded 1676 times)

[Updated on: Tue, 23 October 2007 14:54]

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276065 is a reply to message #276037] Tue, 23 October 2007 16:34 Go to previous messageGo to next message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
The mount point /backup does not seem to be on a shared disk. You could always share it out on node1 and mount it on node2 as (/backup) so that it will have the same name as on node1 (/backup).

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #276645 is a reply to message #276065] Thu, 25 October 2007 15:30 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Hi,
Thanks for the response. I am working to get the /backup shared. I don't have a Unix Admin--so when I can that done I will report to you/forum.

Thanks again

Report message to a moderator

icon14.gif  Re: Archivelog Backup in 2 Node RAC [message #277354 is a reply to message #275374] Mon, 29 October 2007 13:45 Go to previous messageGo to next message
youngb912
Messages: 56
Registered: October 2007
Location: New York
Member
Good afternoono,

I got a successful archive log backup today. As, Ebrain recommended I made the /backup directory "share" and mounted it on node2. Both nodes can now see the backup "to" location.

Thanks a million to"Senior member", Ebrain for the help. I couldn't have gone pass this stage without your help. I can now relax knowing that I am backing up the archivelog.

Again, thanx.

[Updated on: Mon, 29 October 2007 13:46]

Report message to a moderator

Re: Archivelog Backup in 2 Node RAC [message #277385 is a reply to message #277354] Mon, 29 October 2007 20:06 Go to previous message
ebrian
Messages: 2794
Registered: April 2006
Senior Member
Great ! I'm glad to see it worked out for ya !

Report message to a moderator

Previous Topic: rman charecter set to we8mswin1252
Next Topic: controlfile autobackup format
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Mon May 20 12:27:42 CDT 2024
.:: Forum Home :: Blogger Home :: Wiki Home :: Contact :: Privacy ::.