Getting WRDS data into SQLite

As discussed previously, one of the three barriers to getting away from SAS in accounting research is getting the data into an alternative form.

The procedure I used to get data from SAS into an SQLite database involved the following steps:

1. Download the data from WRDS.
2. Convert the data from SAS to comma-separated values (CSV) using StatTransfer.
3. Use Perl to import the CSV data into an SQLite database and do a little work (i.e., indexing) to improve performance.

To illustrate this procedure, I use a subset of the CRSP monthly stock data:

1. Get SAS datasets. If I want to calculate stock returns including delisting returns and compare these returns to the CRSP value-weighted returns, I will need the following SAS datasets from the sm subdirectory of CRSP on WRDS.

A short UNIX shell script did the trick (for me…use your WRDS username in place of mine). Note that by using rsync, I only get files that I don’t have (or have outdated copies of).

2. Convert to CSV. Assuming that the only SAS files in the current directory are the four I downloaded above, I could simply issue the following command:
st "./*.sas7bdat" "./*.csv"

3. Use Perl to create an SQLite database. This is the most involved step (here is the script).

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s