The Torture Database API

The Torture Database is the largest public repository of primary-source documents relating to the Bush administration’s interrogation policies. The database came out of a Freedom of Information Act (FOIA) request filed in 2003, as part of our ongoing effort to uncover how our government used torture in the years following September 11, 2001. Following a court ruling in our favor, the government released more than 130,000 pages of documents on the interrogation program, including memos detailing the techniques authorized by the CIA, autopsy reports of detainees who died in custody, authorizations for the use of torture from the highest levels of government, email threads documenting heroic efforts to stop the use of torture, and more.

The Torture Database API provides access to documents stored at thetorturedatabase.org. It is a RESTful API built on HTTP and:

   -  Uses built-in HTTP capabilities for passing parameters
   -  Has search features comparable to thetorturedatabase.org site
   -  Returns data in JSON or XML formatting

The API was rapidly developed using the Drupal Services module, which provides web service callbacks for data as represented by the Drupal content management system. As such, the structure of the data returned by this API bears a close resemblance to standard Drupal objects.

Formatting Options

The API currently provides two data formatting options, which is determined by the last component in the URL:

JSON formatting:
http://www.thetorturedatabase.org/rest/searchnode/retrieve.json

XML formatting:
http://www.thetorturedatabase.org/rest/searchnode/retrieve

Base URL

All API URLs referenced in this documentation start with one of the following base parts:

To retrieve a list of items for a specific content type (e.g., document):
http://www.thetorturedatabase.org/rest/getnode/retrieve

To retrieve document search results using keywords and filters:
http://www.thetorturedatabase.org/rest/searchnode/retrieve

To retrieve the fields for a specific node by a numeric ID (a node is Drupal term meaning a piece of content on the website):
http://www.thetorturedatabase.org/rest/fullnode/retrieve

Authentication

Since no authentication is required, please be cautious of the number of API calls being made within a short time period. Aggressive use of the API may adversely affect the thetorturedatabase.org site and may lead to the removal of the API for all users.

How to retrieve a list of all data for a certain type

List of data objects: Agency, Authors/Recipients, Document Types, Techniques, Detainee, Document, Incident, Location, Official, Source

Note: By default, only the first 100 results will display for these requests. The page parameter should be passed to view remaining results (e.g. http://www.thetorturedatabase.org/rest/getnode/retrieve?name=[type]&page=1). page=0 by default.

Agency (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=agency

Authors/Recipients (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Authors/Recipients

Document Types (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Document%20Types

Techniques (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Techniques

Detainee (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=detainee

Document (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=document

Incident (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=incident

Location (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=location

Official (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=official

Source (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=source

How to retrieve data for a specific node

A node is a piece of content on the website with nid as the primary key. Each node and its fields can be retrieved in full.

Pass the ID of the node (e.g. 6822):
http://www.thetorturedatabase.org/rest/fullnode/retrieve?nid=6822

The fields returned are all of the custom fields associated with each content type, in addition to some standard Drupal metadata. We’ve attempted to have each field name be self-documenting but please contact [email protected] if the purpose of a field is unclear. References to other objects are done with a numeric ID, and are usually clearly marked with a nid key. References to objects without a nid primary key are marked with a value key and refer to an object with a unique tid primary key. Objects with a tid primary key cannot be retrieved with the fullnode call, as they have no additional fields other than what is provided in the getnode call.

How to search by keyword(s)

The Torture Database API allows you to search the database using similar options as provided through the web interface.

To search by keyword, pass the 'keys' parameter with the searchnode base URL:
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith

Note: Since results are viewed 10 items at a time by default, a page parameter can be set to paginate through the results:
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith&page=2

How to filter keyword results by date, category, and advanced search options

Search filters use the same formatting that is passed to the Apache SOLR search engine. Since the structure of these filters can be opaque, the easiest approach is to copy the filter values from the actual site when using the search/advanced search options. You can copy the part of the URL after ‘filters’ to find the relevant filter options. For example:

http://www.thetorturedatabase.org/search/apachesolr_search/john%20smith?filters=tds_cck_field_doc_date%3A[2009-01-01T00%3A00%3A00Z%20TO%202010-01-01T00%3A00%3A00Z]

Appended to the base URL for searching:

http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith&filters=tds_cck_field_doc_date%3A[2009-01-01T00%3A00%3A00Z%20TO%202010-01-01T00%3A00%3A00Z]

For reference, here are some specific search options:

Filter by Date

Document Date
tds_cck_field_doc_date:[2006-01-01T00:00:00Z TO 2012-01-01T00:00:00Z]
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith&filters=tds_cck_field_doc_date%3A[2006-01-01T00%3A00%3A00Z%20TO%2020012-01-01T00%3A00%3A00Z]

Specific Incident of Abuse
tdm_cck_field_incident_date:[2003-05-01T00:00:00Z TO 2006-05-02T00:00:00Z]

Release Date
tds_cck_field_doc_release_date:[2005-05-01T00:00:00Z TO 2010-05-02T00:00:00Z]

Filter By Category (for multiple values, separate by comma)

Authoring Agency
sm_cck_field_doc_from_agency:35

Officials mentioned
im_cck_field_doc_officials:1700

Detainees Mentioned
im_cck_field_doc_detainees:981

Methods Mentioned
sm_cck_field_techniques:100

Document Type
sm_cck_field_doc_type:649

Advanced Search

Author By Name
sm_cck_field_doc_from:1117

Authoring Agencies
authoring_agencies=31

Recipient By Name
sm_cck_field_doc_to:584

Receiving Agencies
receiving_agencies=31,39

Mentions the Following
im_cck_field_doc_detainees:646

Incident Location
im_cck_field_incident_location:148

Regions
regions:Afghanistan
regions:Iraq
regions:Guantanamo
regions:Everywhere Else