12.4.1. Overview for APQC

Description

afni_proc.py allows researchers to specify their desired steps and options for FMRI processing. The program is designed to be flexible, relatively compact, and to help researchers organize their processing. It is easy to share with others and to modify as processing needs demand it. It builds a processing script based on the user’s options, which is commented and can be checked/read/studied/etc. Several convenience features are taken care of automatically, such as concatenating transforms (e.g., within EPI for motion correction, EPI-> anat, anat -> standard space).

Additionally, afni_proc.py tries to help researchers check their processed data. For example, many intermediate data sets are kept, and several helper scripts are made to show users their data:

• @ss_review_basic

  show basic QC measures (e.g., max motion, number of censored time points, TSNR), in text

• @ss_review_driver

  a script that includes: * “driving” the AFNI GUI to show users their data interactively; * opening up 1dplots to look at motion, censoring, etc.; * showing warning message from processing in the terminal. ... and all of that is commented (!).

• @ss_review_html (new!)

  a command that is automatically run at the end of afni_proc.py processing to make an HTML document full of images+quantities showing information from the processing (combining data from @ss_review_basic and @ss_review_driver with other stuff, too).

Here, describe the usage of the third QC output: the HTML form of APQC.

Background: what afni_proc.py does

The modern AFNI’s afni_proc.py includes the following option:

-html_review_style STYLE : specify generation method for HTML review

        e.g.     -html_review_style pythonic
        default: -html_review_style basic

where STYLE can be one of the following values:

none     : no HTML review pages
basic    : static - time graph images generated by 1dplot
pythonic : static - time graph images generated in python

By default, afni_proc.py will implement the basic form of the review style. Therefore, with no intervention by the user, an HTML doc of QC information will be generated whenever processing your FMRI data.

A related, but slightly prettier, form of QC review is generated with the pythonic option. The only cost is that the user must have Python (version 2.7 or higher) and the matplotlib module. Because this pythonic form of QC review is also slightly more informative, we will just focus on this form here.

With the HTML review style flagged, afni_proc.py will include something like the following few commands after it has finished processing data sets:

# generate html ss review pages
# (akin to static images from running @ss_review_driver)
apqc_make_tcsh.py -review_style pythonic -subj_dir . \
    -uvar_json out.ss_review_uvars.json
tcsh @ss_review_html
apqc_make_html.py -qc_dir QC_$subj

where $subj holds the name of your subject in the processing script. Again, you, Dear Reader, don’t have to do anything with these commands– they will be executed on your behalf automatically. But if you ever want to rerun them, here they be.

After those commands are executed, you will be prompted to finally take action, as afni_proc.py shows you something like the following to open up the freshly created HTML QC doc:

afni_open -b $subj.results/QC_$subj/index.html

... where $subj.results is your directory containing all afni_proc.py results; QC_$subj contains all the APQC results; and “index.html” is the focal document that will be opened. The $subjss will be replaced with your subject name (such as “sub-001” or “sub-753”). The afni_open command will select a browser on your system (e.g., firefox) for opening up the file “index.html”.

The contents and functionality of this APQC document are explored on the subsequent pages here.

Portability: how to share the APQC

As noted above, the directory QC_$subj (e.g., “QC_sub-001” or “QC_sub-216”) contains everything it needed for the QC. Whenever you want to share your QC from processing with another person, or even just move to another computer, be sure to move the entire QC-directory, not just the HTML file or something.

The primary file to note is the index.html file, which is the “target” to open up in a browser. It does not contain embedded images, but instead those sit in sub-directories inside of the QC_$subj directory.

The “media” subdirectory contains all of the images, as well as informative text and JSON files, from which the index.html file is populated. Names and content of these files should not be changed.

There is a “help.html” file contained in each QC-directory, which is accessible through a “HELP” button in the upper right corner of the index.html file.

There is also an apqc*.json file in the main QC-directory. This is initially mainly empty, but it can be populated via the QC buttons in the APQC HTML doc with both ratings for individual QC steps and comments. That is, this file can store the user’s own comments on the data, which can be shared with others (or just recorded for future reference).

Finally, the “extra_info” directory contains a boring CSS style file, as well as well as a dictionary of afni_proc.py-processed files in a JSON.

Saving ratings and comments

As described in the subsequent pages, both ratings and comments of QC steps (generally referred to throughout as “QC blocks”), as well as a final summary evaluation can be stored in the HTML documentation. This can be useful for recording, for instance, why you thought a data set was bad, or a question to ask your boss, or a recipe idea for quiche.

The one downside with having an HTML documentation is that saving changes goes through the browser, and browsers in general are hyper-worried about security and hacks. Because of this, it can’t know convenient features about your computer system like file structure, even when running locally and looking at local files. Therefore, you will have to walk through your file system to the location of your file. If you are editing several QC files in a row and saving ratings/comments, then hopefully at least the files are near each other in the file system, and your browser will open up in the previous directory location, hence shortening the path for your next file save.

Additionally, you will likely want to turn off the feature of your browser to automatically download files to a single place, such as “Downloads” or “Telechargements”.

Those are just some practical considerations and constraints at present, until browsers change.

Finally, we note that the APQC HTML page navigation has been tested most strongly during development with the firefox browser. It should play nicely with any modern, mainstream browser (e.g., Chrome, Explorer, Safari, etc.), but quirks may occur. If you come across any difficulties, please let us know via the Message Board so we can try to address them (however, please note that some issues may not be resolvable except by using a different browser).