Skip to content

Tutorial for Copying Files from a Device

This section covers:

Introduction

Instructions for copying a file from the device will differ depending upon on the device. It can be different on different systems. There are several ways to get files off a device:

  • SCP Pull
  • Windows SMB Get
  • TFTP Push
  • File Output Capture

SCP pull

A typical way to want to fetch a file from a nix system, is to use Secure Copy (SCP) over SSH. To do this Osirium PAM needs to be told the file name to fetch (including the full path).

For example, let's say we want to run a techout task on an F5 Load Balancer, which uses the following command (the nice -20 just helps lessen the impact on the CPU if this is a live device):

1
2
3
4
5
6
[admin@f5-ltm:Active] ~ # nice -20 qkview
Gathering System Diagnostics: Please wait ...
Diagnostic information has been saved in:
/var/tmp/f5-ltm.osirium.net.tgz
Please send this file to F5 support.
[admin@f5-ltm:Active] ~ #

So, we start off with a template that looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
<?xml version="1.0" encoding="utf-8"?>
<devicetemplate>

   <tasks>
      <task display_name="Techout" name="techout" type="status">
         <commands>
            <command response_timeout="300">nice -20 qkview
               <success type="ci_in" value="Please send this file to"/>
               <failed type="default"/>
            </command>
         </commands>
      </task>
   </tasks>

</devicetemplate>

In this example, the qkview support command on an F5 always generates a .tgz file in the /var/tmp directory, with the name of the file set to the hostname of the system. So the next thing that needs to be done is the extract the filename and path needs to be read by storing it using an update_env command.

To get the exact filename and path we can use:

1
ls -1 --color=none /var/tmp/*.tgz

What this command does is lists the files ending in .tgz in the /var/tmp directory, which we know is where the qkview output file is stored. The -1 flag lists the files in long format and one per line. As qkview command overrides the file with the same name each time, we will only get a single line answer.

We also need to use the --color=none flag as by default, in the standard bash shell on the F5, the ls command will colour code archive files (.tgz) in red and it does this by sending the terminal client non-printing control characters that tell it to make the text red. Osirium PAM will see this control characters and include them as part of the filename, which we don't want. By saying we don't want colour, we just get the raw text of the file and path. Like this::

1
2
3
[admin@f5-ltm:Active] ~ # ls -1 --color=none /var/tmp/*.tgz
/var/tmp/f5-ltm.osirium.net.tgz
[admin@f5-ltm:Active] ~ #

This is the filename of the qkview output file we want to copy back to Osirium PAM. So we use this ls command in an update_env command tag to store the filename.

We can now add the following line to the task template:

1
<command format="update_env" attribute="filename" regex_capture="^.*/(.*)$">ls -1 --color=none /var/tmp/*.tgz</command>

The regex_capture attribute above strips the path from the filename, as we only want actual filename.

For more info on regular expressions, see: Regular Expressions

Now we have the filename we can build an SCP command to go fetch the file.

1
2
3
4
<command type="scp" action="retrieve">
   <local_filename>%(local_techout_location)s/%(device_name)s/%(filename)s</local_filename>
   <remote_filename>/var/tmp/%(filename)s</remote_filename>
</command>

We don't need to specify any credentials for the connection as Osirium PAM will automatically use the control account for the device.

A command tag of type scp has two child tags, <local_filename> and <remote_filename>. These are used to define where Osirium PAM should store the file once copied back (local_filename) and the name of the file to go and get (remote_filename).

For a techout, the local file location within Osirium PAM is typically:

1
%(local_techout_location)s/%(device_name)s/

It is then followed by the filename substitution '%(filename)s'

The <remote_filename> tag then defines the path and filename of the file to be fetched from the remote device. In this example we know the file is saved to '/var/tmp' so we manually define this and the let Osirium PAM fill in the filename it read with the 'update_env' earlier::

1
/var/tmp/%(filename)s

This is then enough information for Osirium PAM to connect to the device, using the devices control account over SCP, fetch the file /var/tmp/%(filename)s and save it locally within Osirium PAM file system at %(local_techout_location)s/%(device_name)s/%(filename)s.

The final full template then looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
<?xml version="1.0" encoding="utf-8"?>
<devicetemplate>

   <tasks>
      <task display_name="Techout" name="techout" type="fileout">
         <commands>
            <command response_timeout="300">nice -20 qkview
               <success type="ci_in" value="Please send this file to"/>
               <failed type="default"/>
            </command>
            <command format="update_env" attribute="filename" regex_capture="^.*/(.*)$">ls -1 --color=none /var/tmp/*.tgz</command>
            <command type="scp" action="retrieve">
               <local_filename>%(local_techout_location)s/%(device_name)s/%(filename)s </local_filename>
               <remote_filename>/var/tmp/%(filename)s</remote_filename>
            </command>
         </commands>
      </task>
   </tasks>

</devicetemplate>

Windows SMB get

To fetch files from a Windows computer (workstation or server) Osirium PAM needs to connect to a Windows SMB (aka CFIS) file share on the server and pull the files. This share is automatically setup by Osirium PAM with a pre-configured task called verify_and_create_share.

This task looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
<task name="verify_and_create_share" type="status">
   <commands>
      <command>mkdir %(share_directory)s\%(device_name)s
         <success type="ci_in" value="already exists"/>
         <success type="ci_match" value=""/>
         <failed type="default"/>
      </command>
      <command>net share %(share_root)s%(device_name)s=%(share_directory)s\%(device_name)s /grant:%(logonname)s,full
         <success type="ci_in" value="The name has already been shared"/>
         <success type="ci_in" value="was shared successfully"/>
         <failed type="default"/>
      </command>
      <command>echo y|cacls %(share_directory)s\%(device_name)s /G %(logonname)s:F administrators:F
         <success type="ci_in" value="processed dir"/>
         <failed type="default"/>
      </command>
   </commands>
</task>

What this does is creates a directory to store files in that need to be pulled from the Windows device by Osirium PAM. It then shares this directory and allows the control account user to access the share. The default 'Administrators' group is also given access. This task uses two configuration strings which are also pre-configured. These are:

1
2
3
4
<configurations>
   <configuration fieldname="share_directory" fieldtype="device" value="%SYSTEMDRIVE%\osirium"/>
   <configuration fieldname="share_root" fieldtype="device" value="osirium_"/>
</configurations>

These are the defaults, they can be changed if required.

Whenever a task runs that need to copy a file from a Windows device, the 'verify_and_create_share' task needs to be called as a subtask.

This will either verify the folder and share exist, or it will create them if they don't. Either way it means the device is setup so Osirium PAM can access the files on it it needs too.

This subtask is called with:

1
<command type="call" taskname="verify_and_create_share"/>

Once this subtask has been called, the parent task can then continue to perform its commands to create an output file. Then at the end, the file can be copied from the device.

As a worked example, let's create a task that will output the IP configuration of a device to a file and then copy this file back to Osirium PAM. We'll use the normal windows ipconfig command to output the IP configuration.

First we call the verify_and_create_share task. Then we run the 'ipconfig' command and pipe the output to a file which we can then copy back, like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
   <?xml version="1.0" encoding="utf-8"?>
   <devicetemplate>

      <tasks>
         <task display_name="Capture IP Configuration" name="capture_ip" type="status">
            <commands>
               <command type="call" taskname="verify_and_create_share"/>
               <command>ipconfig >> %(share_directory)s\%(device_name)s\ipconfig.txt</command>
            </commands>
         </task>
      </tasks>

   </devicetemplate>

The command output is echoed to a file located in the directory used for the share, as detailed earlier. We don't need to look for specific text like Windows IP Configuration because the output is echo'ed into the file, so we can simply expect a standard blank prompt to respond back.

All that is left now is for Osirium PAM is to fetch the file back. We do this with an smb.get command like this:

1
2
3
   <command type="local_shell" command="smb.get" source="//%(share_root)s%(device_name)s/ipconfig.txt" destination="%(local_filename)s">
       <local_filename>%(filestore_location)s/%(device_name)s/ipconfig_%(O_TASK_UTC_STARTTIME)s.txt</local_filename>
   </command>
The final full split template file then looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
   <?xml version="1.0" encoding="utf-8"?>
   <devicetemplate>

      <tasks>
         <task display_name="Capture IP Configuration" name="capture_ip" type="status">
            <commands>
               <command type="call" taskname="verify_and_create_share"/>
               <command>ipconfig >> %(share_directory)s\%(device_name)s\ipconfig.txt</command>
               <command type="local_shell" command="smb.get" source="//%(share_root)s%(device_name)s/ipconfig.txt" destination="%(local_filename)s">
                  <local_filename>%(filestore_location)s/%(device_name)s/ipconfig_%(O_TASK_UTC_STARTTIME)s.txt</local_filename>
               </command>
            </commands>
         </task>
      </tasks>

   </devicetemplate>

The <local_filename> tag is then used to set the filename where the file will be stored on Osirium PAM. The substitution %(O_TASK_UTC_STARTTIME)s can be used to add a unique timestamp to the file. The file is made available to download from the usual files page.

File output capture

Some devices might not allow files to be copied using SCP or SMB. An alternative method is to capture text that is outputed to the communication channel being used, i.e. SSH.

Note

In some cases this method can NOT be used for creating a backup of a device. Some devices, when asked to show the full system configuration on the screen, star out passwords and other security sensitive data and therefore would make using a capture of this data useless as a backup.

Consider we want to create a techout of diagnostic information from a Cisco device. We would use the Cisco command::

1
   show tech-support

This generates several pages of output which we wish to capture and store as a file on Osirium PAM. To do this we would use the following task:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
   <?xml version="1.0" encoding="utf-8"?>
   <devicetemplate>

      <tasks>
         <task display_name="Techout" name="techout" type="status">
            <commands>
               <command format="text_capture">show tech-support
                  <local_filename>%(local_techout_location)s/%(device_name)s/techout_%(O_TASK_UTC_STARTTIME)s.txt</local_filename>
                  <success type="default"/>
               </command>
            </commands>
         </task>
      </tasks>

   </devicetemplate>

Because all output from the command is captured, good or bad, there is no need to have a success test. We can simply use a <success> tag set to default on the command.

TFTP push

A less often used method, due to it's insecure unencrypted nature, is to do a TFTP transfer. However, on some old and on some basic devices, this can be the only way to get a file or backup file off a device.

For example, to get an old Cisco device to copy the running-config to a file on Osirium PAM using TFTP use the following task:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
   <?xml version="1.0" encoding="utf-8"?>
   <devicetemplate>

      <tasks>
         <task display_name="Backup" name="backup" type="status">
            <commands>
               <command type="tftp" random_filename="%s.cfg">copy system:/running-config tftp://%(osirium_address)s/%(random_filename)s
                  <success type="ci_in" value="bytes copied"/>
                  <failed type="default"/>
                  <local_filename>%(local_backup_location)s/%(device_name)s/backup_%(O_TASK_UTC_STARTTIME)s.txt</local_filename>
               </command>
            </commands>
         </task>
      </tasks>

   </devicetemplate>

The Osirium PAM creates a random filename and uses this to copy the file back. The TFTP server on Osirium PAM only runs whilst it is waiting for this file to be copied back and will only accept the specific random filename from that specific device. When the file is then stored within Osirium PAM, the <local_filename> tag updates the random filename to the one defined.