Loading...

Login
Reset Password

NDAR provides a single access to de-identified autism research data. For permission to download data, you will need an NDAR account with approved access to NDAR or a connected repository (AGRE, IAN, or the ATP). For NDAR access, you need to be a research investigator sponsored by an NIH recognized institution with federal wide assurance. See Request Access for more information.

Warning Notice

This is a U.S. Government computer system, which may be accessed and used only for authorized Government business by authorized personnel. Unauthorized access or use of this computer system may subject violators to criminal, civil, and/or administrative action. All information on this computer system may be intercepted, recorded, read, copied, and disclosed by and to authorized personnel for official purposes, including criminal investigations. Such information includes sensitive data encrypted to comply with confidentiality and privacy requirements. Access or use of this computer system by any person, whether authorized or unauthorized, constitutes consent to these terms. There is no right of privacy in this system.

You have logged in with a temporary password. Please update your password. Passwords must contain 8 or more characters and must contain at least 3 of the following types of characters:

Subscribe to our mailing list

Mailing List(s)
Email Format

You are now leaving the National Database for Autism Research (NDAR) web site to go to:

Click on the address above if the page does not change within 10 seconds.

Disclaimer

NDAR is not responsible for the content of this external site and does not monitor other web sites for accuracy.

Switch User

Cloud Access Overview

The mission of the National Institute of Mental Health Data Archive (NDA) is to make research data available for reuse. Data collected across projects can be aggregated and made available using the GUID, including clinical data, and the results of imaging, genomic, and other experimental data collected from the same participants. In this way, separate experiments on genotypes and brain volumes can inform the research community on the over one hundred thousand subjects now contained in the NDA. The NDA’s cloud computation capability provides a framework in support of this infrastructure.

 

How does it work?

The NDA holds and protects rich datasets (fastq, brain imaging) in object-based storage (Amazon S3). To facilitate access, the NDA supports the deployment of packages (created through the NDA Query tools) to an Amazon Web Service Oracle database. Originally developed for the National Database for Autism Research (NDAR), and so called miNDAR (miniature NDAR), these databases contain a table for each data structure in a package. Associated raw or evaluated data files are available via read-only access to NDA’s S3 objects. Addresses for those objects in the associated package are provided in the miNDAR table titled S3_LINKS. By providing this interface, the NDA envisions real-time computation against rich datasets that can be initiated without the need to download full packages. Furthermore, a new category of data structure has been created called "evaluated data." Tables for these structures will be created for each miNDAR, allowing researchers using NDA cloud capabilities and computational pipelines to write any analyzed data directly back to the miNDAR database. This will enable the NDA to make this data available to the general research community when appropriate.

 

miNDARs can also be populated with your own raw or evaluated data and uploaded directly back into the NDA for a streamlined data submission directly from a hosted database.

 

How do I get started?

To begin, email NDAHelp@mail.nih.gov and request that cloud access be added to your account. Once your request is approved, the option to launch packages to a cloud hosted database will be available during package creation. You can deploy previously generated packages as well as new ones.

 

Initiating miNDAR

To move NDAR data to Oracle, first create a package in NDAR. Then, following registration, enter the package id and credentials requested on the miNDAR tab. This will start the miNDAR creation process, which takes approximately 10 minutes. Once created, the miNDAR connect details will be emailed to you, and can be used to establish a connection with your credentials.

 

Files

File data that is usually included in a package download will now be accessible via S3. Each package will have a table “S3_LINKS” which contains URIs for all objects in that package. Using direct calls to Amazon Web Service's S3 API, a third party tool, or client libraries, data from these objects can be streamed or downloaded. 

 

Temporary Keys

For security purposes temporary AWS credentials  are needed to access the S3 Objects. Temporary credentials are issued by authenticating with a web service using your NDAR username and password.  AWS credentials can be obtained directly from the web service (see examples on our GitHub page) or from the download manager, which is available in both a GUI and command line version.

For the GUI version, go to the 'Tools' menu and select 'Generate AWS Credentials'.  

For the command line download manager, use the following syntax: 

java -jar downloadmanager.jar --username user --password pass --g 

For help with the command line download manager, use the following switches: -h, --help

The web service provides temporary credentials in three parts:

  • an access key, 
  • a secret key, 
  • and a session token

All three parts are needed in order to authenticate properly with S3 and retrieve data.  

Additionally the web service provides an expiration timestamp for the token in YYYY-MM-DDTHH:MM:SS-TZ format (TZ=HH:MM).  New keys can be retrieved at any time. A service oriented approach allows for implementation of pipeline procedures which can request new keys at the appropriate stage of data processing. 

 

Additional Help

Please see our Cloud Tutorials for a video demonstration of how to create a miNDAR, how to generate temporary security credentials, and how to use these to retrieve data.  Please contact the Help Desk with any questions.