Grab a Python dictionary by the tail!

The problem I have to solve is to take a history log from the Library system and create some Excel friendly files to look at he various transactions. there are plenty of tools in the system but they can’t give me the flexibility to get the data into Excel for charting or other manipulations. I decided to use Python to break the data up into a usable form.

I decide to unpack the log into separate files, using the command from the log entry to name the file. The command is in the log as a two character code, too ambiguous for others to use the files after unpacking. So I looked for a method of using the two character code to get the description. A Python dictionary looked like the best way to accomplish this.

I grabbed the document from support with the command code and description and worked modified it into a text file with one code and description, AV_Add_Item, per line. You will notice the _ used wherever a space should be. I have found scripts and programs really don’t like spaces in the file names so this is a fix for that.

After opening the command file for reading, creating the dictionary is accomplished with for in loop using these steps.

  1. Read in on line of the command file.
  2. Get the length of the line.
  3. Load the command into it’s variable.
  4. Load the command description into its variable.
  5. Append the dictionary with the new entry.
  6. Continue with the next line.

After creating the dictionary to use for the file name, I set up a for in loop to test each log entry and move that line to a file name of it’s own. Because I don’t know which command will be next I use the command code to get the file name, then loop through these steps:

  1. Open the history log file for reading.
  2. Read in one line of the log history file
  3. Prime the file name variable with c:\data\.
  4. Get the command code from the log entry line.
  5. Use that code to get the command description.
  6. Combine the base file name from step one with the description giving us c:\data\Add_Item.
  7. Open that file for appending.
  8. Write the log entry to the file.
  9. Close the file and process the next line.

I ended up with 60+ files reflecting the transaction history of one year.

This is what a transaction history line looks like.

E201301030937130033R ^S78CVFFJCIRC^FEMAIN^FcNONE^FWJCIRC^NQ36034000214160^UOPRINTCHECK^Uf0652^dC3^rsY^jz1^Fv200000^^O

Below is the program with lots of notation.

# Created on a win system with IDLE
# February 2016

# The plan is to read in on line of the Sirsi history log
# find the command code and save each line to a separate file
# named for the command code.

# Setup the variables

cc = "" # Holds the command code used as the matching
cd = "" # Holds the command description to place in dictionary
command = {} # Holds the dictionary of command codes and real name
ccw = "" # Holds working code and description to build command dictionary

working = "" # Holds the log file line we are working on
ccw = "YY" # Holds the command code for matching
cdw = "XX" # Holds the command description
ccfname = "ZZ" # Holds the current file name to export to

# Load the command dictionary

# Open the file with the commands and definitions for reading
record = open("c:/data/cc.txt", "r")

# Cycle through the file building the dictionary with command codes and the definition
# The definition has been altered to have an _ in place of a space to allow for use
# as the file name for the command separation

for ccw in record:

 cl = len(ccw) # Get the length of the line
 cc = ccw[:2] # Put the command code in cc, position 1-2
 cd = ccw[3:(cl-1)] # Put the description,position 3 to (cl-1), in cd without the new line \n
 command[cc] = cd

record.close() # Close the file

# Open the history log file for reading
record_log = open("c:\data\log.txt","r")

# cycle through the history log file to extract each command to it's own file
for working in record_log: 
 ccfname = "c:/data2/" # Setup base folder for data files
 ccw = working[25:27] # Get the command code
 cdw = command[ccw] # Get the definition from the dictionary to use as the fil ename
 ccfname = ccfname + cdw # Concatenate the base folder and the command file name

 output = open(ccfname,"a") # Open the specific file for appending
 output.write(working) # Write out the log file data in working
 output.close() # Close this file, for now

print "All Done" # Let you know when done
record_log.close() # Close the file


About jcoffey
This entry was posted in commentary. Bookmark the permalink.