The Department of Homeland Security (DHS) just released 1,400 pages of emails and other internal records concerning the agency’s role in advising the Department of Justice and Office of Director of National Intelligence (ODNI) on changes made to the National Counterterrorism Center (NCTC) guidelines used to collect and retain data about potential terrorist threats.

DHS posted the documents to its Freedom of Information Act (FOIA) reading room Thursday afternoon. The records include proposed talking points about the updated NCTC guidelines, which were implemented in March 2012. DHS said the ACLU and Wall Street Journal are the original requesters of the records. 

[UPDATE: After this report was posted, Wall Street Journal reporter Julia Angwin, who is currently writing a book about privacy in the digital age, tweeted that most of the records DHS posted to its reading room Thursday were turned over to her last year. But it appears DHS released additional emails, said Angwin, whose December 2012 investigative report about the changes to NCTC guidelines, based on some of these documents, can be found here.] 

The documents are so heavily redacted it’s impossible to determine what the talking points said and what advice DHS had offered DOJ and ODNI in drafting the new guidelines. The documents show that DHS’s Intelligence & Analysis and Civil Rights and Civil Liberties Division spent more than a year working with the agencies.

Last March, Attorney General Eric Holder and Director of National Intelligence James Clapper issued a joint statement announcing that they were updating NCTC’s November 2008 guidelines “that governed NCTC’s access, retention, use, and dissemination of “terrorism information” contained within federal datasets that are identified as also including non-terrorism information and information pertaining exclusively to domestic terrorism.”

The most controversial aspect of NCTC’s updated guidelines was that the agency now had the ability to retain data on U.S. persons for five years—even if the individual is not suspected of being involved in terrorist activities—instead of 180 days under the November 2008 guidelines.

In a July 6, 2011 email written by DHS Associate General Counsel Matthew Kronisch that he sent to DHS General Counsel Ivan Fong and DHS Under Secretary for Intelligence & Analysis Caryn says, “I have inserted two new bullets at the top of the paper to clarify why the new guidelines are needed.” 

But the “paper” included in the batch of records released by DHS is completely redacted.

Holder and Clapper pointed to the intelligence community’s missed opportunity to connect the dots about “underwear bomber” Umar Farouk Abdulmutallab, who attempted to bomb a commercial aircraft on Christmas Day 2009 to justify the increase in retaining data on Americans.

“Following the failed terrorist attack in December 2009, representatives of the counterterrorism community concluded it is vital for NCTC to be provided with a variety of datasets from various agencies that contain terrorism information,” Clapper said when the new guidelines were rolled out. “The ability to search against these datasets for up to five years on a continuing basis as these updated Guidelines permit will enable NCTC to accomplish its mission more practically and effectively than the 2008 Guidelines allowed.”

The documents released by DHS shows that the agency weighed in on the retention of data, in one instance pertaining to the “Refugee, Asylum and Parole System (RAPS)” dataset.

According to a June 1, 2011 email, that included an attachment titled, “Progress on NCTC Engagement Over Past Two Years 05-23-11.docx; Information Sharing,” an unnamed DHS official wrote:

“Last week I sent you this email with RAPS comments and a suggestion about briefing on the retention period/broader use issue, with the goal of having some answers for NCTC in my next meeting with them, and having Margo prepared for the ‘information sharing’ S1meeting which had just shown up on her calendar.”

The rest of the email is redacted.

One document from the batch of records is an unclassified, for official use only document titled, “System Description of DHS Datasets.” The document provides insight into five separate datasets DHS uses to collect and retain information about individuals: “Arrival/Departure Information System (ADIS),” “Advance Passenger Information System (APIS),” “Electronic System for Travel Authorization (ESTA),” “Refugee, Asylum and Parole System (RAPS),” and “Student Exchange and Visitor Information System (SEVIS).”

The document goes on to say that information from these datasets is being shared  “under 6/28/11 MOU [memorandum of understanding].” DHS had previously said the agency was sharing entire databases with the intelligence community that contained this type of information.