Backup & Recovery manager for Avamar, Networker and Datadomain.

New EMC centralized monitoring tool for Avamar Networker and Datadomain only. 

how to find Avamar serial number for utility and storage nodes

For Avamar Datastore Generation 4 nodes

Avamar Datastore Generation 4 hardware consists of two different types of Dell nodes (R510 and R710). A new hardware information retreival system is used and has a different command that will work on all Gen4 nodes.

To retrieve the serial number on a node, login as root and issue the following command:

/usr/bin/ipmitool fru print 0 " grep "Product Asset Tag" " sed "s/^.*: *\(.*\)/\1/"

This command must be run as root user or else you will see an error such as:

"Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory"

The following mapall command is run from the utility node (run as admin with dpnid key loaded) will retrieve the serial numbers from all of the nodes in the grid.

mapall --noerror --user=root --nodes=all+ --quiet 'echo `hostname -i`": "`/usr/bin/ipmitool fru print 0 " grep "Product Asset Tag" " sed "s/^.*:\s\(.*$\)/\1/"`'

Sample output:

10.6.240.193: NNG05104310280_100-580-617_A04
10.6.240.194: NNG05104310279_100-580-617_A04
10.6.240.195: NNG05104310284_100-580-617_A04
10.6.240.196: NNG05104310287_100-580-617_A04

The node serials in the example above start with NNG up to the first underscore.

The 100-580-617_A04 part is the EMC part/revision number.

Avamar unix client side restore, add noticketrequired for the user created. Ldap

Admin@avamar> avmgr logn --id=rman_user@/rman --password=rman_user

1 Request succeeded

32817 privilege level (enabled,read,backup,mclogin)

2 block type (directory)

Admin@avamar>avmgr chgv --path=/rman  --u=rman_user --pv=’enabled,read,backup,mclogin,noticketrequired’ –ud=’avamar’

1 Request succeeded

36913 privilege level (enabled,read,backup,noticketrequired,mclogin)

2 block type (directory)

Man-In-The-Middle attack: “REMOTE HOST IDENTIFICATION HAS CHANGED!”

If you receive a message referencing a possible Man-In-The-Middle attack:
“REMOTE HOST IDENTIFICATION HAS CHANGED!”

Remove the IP address in question from "/home/admin/.ssh/known_hosts"

Redirected restore disabled for all non-administrators in Avamar 6.1 and above. You can enable by following this procedure.

Avamar 6.1, Redirect restore is disabled by default for non-administrator users and needs to be enabled from the avamar server.

1-      Login to avamar as root

2-      Run the commands in red without #

3-      # cd /usr/local/avamar/var/mc/server_data/prefs/

4-      # cp mcserver.xml mcserver.xml_orig

5-      Open the file mcserver.xml, you can use vi

6-      Search for the entries below

 entry key="admin_can_direct_restores" value="false" 

 entry key="restore_admin_can_direct_restores" value="false" 

7-      Change both entries to true

 entry key="admin_can_direct_restores" value="true" 

 entry key="restore_admin_can_direct_restores" value="true" 

8-      Save the file

9-      Need to restart the mcs (All backup and restore activities will be cleared)

10-   # su - admin

11-   # ssh-agent bash

12-   # ssh-add .ssh/dpnid

13-   # dpnctl stop mcs

14-   # dpnctl start mcs

15-   # dpnctl start

If checkpoints fails with DDR error

root@Avamar_server:~/#: cplist
cplist: ERROR: ddrmaint: <4750>Datadomain get checkpoint list operation failed.

cp.20120926170740 Wed Sep 26 12:07:40 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120926185155 Wed Sep 26 13:51:55 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120927170726 Thu Sep 27 12:07:26 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120927184726 Thu Sep 27 13:47:26 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120928170738 Fri Sep 28 12:07:38 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120928190034 Fri Sep 28 14:00:34 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120929170731 Sat Sep 29 12:07:31 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120929184944 Sat Sep 29 13:49:44 2012 invalid --- --- nodes 5/5 stripes 77848
cp.20120930170728 Sun Sep 30 12:07:28 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20120930184916 Sun Sep 30 13:49:16 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20121001170739 Mon Oct 1 12:07:39 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20121001184754 Mon Oct 1 13:47:54 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20121002170739 Tue Oct 2 12:07:39 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20121002190634 Tue Oct 2 14:06:34 2012 invalid --- --- nodes 5/5 stripes 77850
cp.20121003170734 Wed Oct 3 12:07:34 2012 invalid --- --- nodes 5/5 stripes 77852
cp.20121003185200 Wed Oct 3 13:52:00 2012 invalid --- --- nodes 5/5 stripes 77852
cp.20121004170736 Thu Oct 4 12:07:36 2012 invalid --- --- nodes 5/5 stripes 77852
cp.20121004190124 Thu Oct 4 14:01:24 2012 invalid --- --- nodes 5/5 stripes 77852
cp.20121005170734 Fri Oct 5 12:07:34 2012 invalid --- --- nodes 5/5 stripes 77854
cp.20121005190516 Fri Oct 5 14:05:16 2012 invalid --- --- nodes 5/5 stripes 77854
cp.20121006170757 Sat Oct 6 12:07:57 2012 invalid --- --- nodes 5/5 stripes 77860
cp.20121006185724 Sat Oct 6 13:57:24 2012 invalid --- --- nodes 5/5 stripes 77860
cp.20121007170750 Sun Oct 7 12:07:50 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121007185441 Sun Oct 7 13:54:41 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121008170750 Mon Oct 8 12:07:50 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121008185438 Mon Oct 8 13:54:38 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121009170757 Tue Oct 9 12:07:57 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121009190138 Tue Oct 9 14:01:38 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121010170754 Wed Oct 10 12:07:54 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121010190509 Wed Oct 10 14:05:09 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121011170740 Thu Oct 11 12:07:40 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121011185927 Thu Oct 11 13:59:27 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121012170652 Fri Oct 12 12:06:52 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121012185325 Fri Oct 12 13:53:25 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121013170656 Sat Oct 13 12:06:56 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121013185554 Sat Oct 13 13:55:54 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121014170505 Sun Oct 14 12:05:05 2012 invalid --- --- nodes 5/5 stripes 77862
cp.20121014192822 Sun Oct 14 14:28:22 2012 invalid --- --- nodes 5/5 stripes 77862

Solution:

The /usr/local/avamar/var/ddr_info file had been zero’d out.  need to recreate ddr_info file, and wait for the extra checkpoints to be removed after next cp and hfschecks.

to check DIMM errors on avamar grid.

login as admin

add ssh keys

admin@avamar_grid_name:~/>: mapall --all 'omreport chassis memory | egrep "^Connector|^Status"' 2>&1 | more Using /usr/local/avamar/var/probe.xml