TicTimer software for measuring tic suppression

Jonathan K. Black; Jonathan M. Koller; Kevin J. Black

doi:10.12688/f1000research.12327.2

Home Browse TicTimer software for measuring tic suppression

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Software Tool Article

Revised

TicTimer software for measuring tic suppression

[version 2; peer review: 2 approved]

Jonathan K. Black¹, Jonathan M. Koller^2,3, Kevin J. Black ^2-5

PUBLISHED 22 Dec 2017

Author details Author details

¹ Ira A. Fulton College of Engineering and Technology, Brigham Young University, Provo, Utah, 84602, USA
² Department of Radiology, Washington University School of Medicine, St. Louis, Missouri, 63110, USA
³ Department of Psychiatry, Ira A. Fulton College of Engineering and Technology, Brigham Young University, St. Louis, Missouri, 63110, USA
⁴ Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, 63110, USA
⁵ Department of Neurology, Washington University School of Medicine, St. Louis, Missouri, 63110, USA

Jonathan K. Black
Roles: Methodology, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan M. Koller
Roles: Methodology, Software, Writing – Review & Editing

Kevin J. Black
Roles: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Tics collection.

Abstract

Woods and Himle developed a standardized tic suppression paradigm (TSP) for the experimental setting, to quantify the effects of intentional tic suppression in Tourette syndrome. The present article describes a Java program that automates record keeping and reward dispensing during the several experimental conditions of the TSP. The software can optionally be connected to a commercial reward token dispenser to further automate reward delivery to the participant. The timing of all tics, 10-second tic-free intervals, and dispensed rewards is recorded in plain text files for later analysis. Expected applications include research on Tourette syndrome and related disorders.

Keywords

tic disorders,Tourette syndrome, reward, reinforcement, psychology, software

Corresponding author: Kevin J. Black

Competing interests: No competing interests were disclosed.

Grant information: Software development and manuscript preparation were funded in part by the U.S. National Institutes of Health (NIH), grant numbers K24 MH087913, R21 NS091635, and R01 MH104030.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2017 Black JK et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Black JK, Koller JM and Black KJ. TicTimer software for measuring tic suppression [version 2; peer review: 2 approved]. F1000Research 2017, 6:1560 (https://doi.org/10.12688/f1000research.12327.2) First published: 24 Aug 2017, 6:1560 (https://doi.org/10.12688/f1000research.12327.1) Latest published: 22 Dec 2017, 6:1560 (https://doi.org/10.12688/f1000research.12327.2)

Revised Amendments from Version 1

This revision addresses questions raised by the reviewers. The introduction is expanded to clarify that the software does not detect the tics; rather, it facilitates an observer’s recording their timing. The introduction and conclusion also clarify that the intended use of the software is primarily for research in the office setting. Future adaptations of the software for clinical utility are possible, and discussed in conclusions. Finally, we now provide in the implementation section a direct link to the GitHub page for the python script that can be used to parse the log files.

See the authors' detailed response to the review by Patrick Haggard
See the authors' detailed response to the review by Danielle C. Cath

Introduction

Woods and Himle developed a tic suppression paradigm (TSP) that could be used in the experimental setting to demonstrate and quantify the effects of intentional tic suppression on tic rate in Tourette syndrome (TS) and other tic disorders^1–5. In this paradigm, each participant is observed during several experimental conditions, baseline and differential reinforcement of zero-rate ticcing (DRO), and sometimes also verbal instruction to suppress tics and/or noncontingent reinforcement (NCR).

In the course of conducting a longitudinal study of children with Provisional Tic Disorder⁶, we found that tic suppression is seen within the first few months after a child’s first tic⁷. We also found that the TSP required substantial investigator effort, and we started writing software with the following goals:

automated tic counting, timing and record-keeping;
automated reward delivery in the DRO condition;
automated reward delivery in the NCR condition.

The overall motivations included not only convenience but also improvement in accuracy. Note that “automated tic counting” here refers to minimizing an expert observer’s record-keeping (simply pushing a button for each tic observed), not to machine detection of the tics. This software, and optional connection to hardware, were intended for use in the research session with the TSP as described above; later in Conclusions we describe possible adaptations for other settings. We present the software here to facilitate its use by others.

Methods

Implementation

TicTimer first has the user set up the details for a session, then it runs a clock for the specified session time while writing significant events to a log file. The program writes a line to the log file for each of the following events: session started and ended, tic detected, ten seconds passed without tics, and reward dispensed. Each line includes the time of the event. By parsing through each line in a log file, a python script (also available on GitHub) can extract and summarize the key data.

The hardware allows reward tokens to be dispensed automatically to the study participant. One end of the long cable enters the token dispenser and its two wires attach to the two pins in the Passive Connection Panel that, when shorted, trigger release of a reward token. The Student Trainer Interface box provides power to the token dispenser box and a remote pushbutton for manually triggering token release. The other end of the long cable connects to the two normally open pins on the relay inside of the small plastic box. The USB to TTL serial cable attaches to the input pins on the relay, with the USB end of the cable leaving the box to attach to the computer running TicTimer. Figure 1 shows the final assembly.

Figure 1. From left to right: Token dispenser box, Student Trainer Interface with external pushbutton, box containing relay module, connected via USB cable to laptop running TicTrainer.

Parts list:

Med Associates token dispenser box, Part #ENV-703
Med Associates SG-595 Student Trainer Interface
Med Associates SG-215D3 Passive Connection Panel
5V Relay module (SainSmart part # 20-018-100)
USB to TTL serial cable (Adafruit part # 954)
Plastic project enclosure box (Velleman part # WCAH2855)
~6m cable with at least one pair of wires free

Operation

System software requirements are Java 8 and RXTX for Java, a library for serial port communication. Binaries for Windows and Linux are provided by fizzed.com. Python 3 was used for the log file reader script.

The program can be run with or without the relay and USB cable. If the hardware is set up and connected to the computer, the program can start in “link mode.” Otherwise, TicTimer can still be run in “button mode,” in which the automatic reward system is replaced by a human who presses the push button attached to the Student Trainer Interface when prompted by a beep and a red flash on the computer screen.

The following procedure applies to both reward modes. First, the user presses “Setup” to choose which type of session is being run and to specify where the session log should be saved. For the NCR condition, the user is also prompted to identify the log file from a previously completed DRO session in the same subject (which provides the timing for the rewards dispensed in the NCR condition). Once setup is completed, the session can be started. During a session, the person observing the subject records tics by clicking the “Tic Detected” button or by pressing “T” or the space bar. If the session type includes rewards (DRO and NCR), they are dispensed appropriately. The session ends when the predetermined time elapses or when the user ends it manually by pressing “End Session” or by closing the window.

If a session is ended manually and restarted, the new session log will be appended to the old one unless a new file was chosen in setup. If a log file contains multiple sessions, only the last session will be used by the NCR mode (which requires a DRO session file in setup) and the data reader script.

To summarize the data from a TicTimer log file, run the accompanying python script (TT_Data.py) with one or more log files as arguments. For each log file given, the script reports the length of the session and the number of tics, 10-second tic-free intervals, and rewards dispensed during the session. The choice of 10 seconds as the duration of rewarded tic-free intervals was made to replicate Woods and Himle’s (2004) original methods, and because we have used that duration in all of our own studies.

Use cases

We have provided 8 sample session log files as Supplementary Files (Supplementary files 1–Supplementary file 8) (subject100_session*_TicTimer_log.txt). These are examples of the files that TicTimer creates during a session. Each line in a log file contains an event and the time at which it occurred. Log files are written in plain English, so they can be read directly if desired. These sample data originated from a participant in the study described by Greene et al.⁷, but all identifying data were removed and these files no longer comprise human subjects data.

The file TT_Data_output.txt (Supplementary file 9) contains output from the python script, summarizing those session files. This output is again in plain text, reporting the session length and the number of each type of event recorded for each log file.

Conclusions

The TicTimer program, now connected to the reward token dispenser, has simplified implementing the TSP and improved the accuracy of reward delivery (given inevitable limitations of human attention and response time in button mode). The software, while designed for our purposes in tic disorder research, may find other uses. The most obvious of these may be for research on traditional habit disorders; for instance, hair pulling and skin picking appear in the “Obsessive-compulsive and related disorders” section of DSM-5⁸. The most obvious application to the clinical setting may be in documenting suppression ability in the office, before and after treatment. However, we have created loosely related web-based software designed for a potential clinical application¹⁰. Another potential future modification would be to add machine detection of tics, e.g. by online video analysis or accelerometry; such an improvement would be quite welcome but is difficult to reduce to practice.

Software availability

The source code is available on GitHub under a BSD 3-clause license.

The current release is available on Zenodo, at DOI 10.5281/zenodo.837884.

Competing interests

No competing interests were disclosed.

Grant information

Software development and manuscript preparation were funded in part by the U.S. National Institutes of Health (NIH), grant numbers K24 MH087913, R21 NS091635, and R01 MH104030.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgments

We gratefully acknowledge the initial tic suppression studies by Woods and Himle.

Supplementary material

The first 8 supplementary files are the output from running 8 TicTimer sessions in one subject.

Supplementary File 1: Baseline Log 1

This file came from the first baseline session.

Click here to access the data.

Supplementary File 2: Verbal Log 1

Output from the first session with only verbal instruction to resist tics.

Click here to access the data.

Supplementary File 3: DRO Log 1

Output from the first DRO session.

Click here to access the data.

Supplementary File 4: NCR Log 1

Output from the first NCR session.

Click here to access the data.

Supplementary File 5: Verbal Log 2

Output from the second session with only verbal instruction to resist tics.

Click here to access the data.

Supplementary File 6: Baseline Log 2

Output from the second baseline session.

Click here to access the data.

Supplementary File 7: DRO Log 2

Output from the second DRO session.

Click here to access the data.

Supplementary File 8: NCR Log 2

Output from the second NCR session.

Click here to access the data.

Supplementary File 9: Data Reader Script Output

Output from the command "python TT_Data.py *_log.txt" in the directory containing the 8 sample log files. It summarizes the data contained in those files.

Click here to access the data.

Faculty Opinions recommended

References

1. Woods DW, Himle MB: Creating tic suppression: comparing the effects of verbal instruction to differential reinforcement. J Appl Behav Anal. 2004; 37(3): 417–20. PubMed Abstract | Free Full Text
2. Himle MB, Woods DW: An experimental evaluation of tic suppression and the tic rebound effect. Behav Res Ther. 2005; 43(11): 1443–51. PubMed Abstract | Publisher Full Text
3. Himle MB, Woods DW, Bunaciu L: Evaluating the role of contingency in differentially reinforced tic suppression. J Appl Behav Anal. 2008; 41(2): 285–9. PubMed Abstract | Free Full Text
4. Lyon GJ, Samar SM, Conelea C, et al.: Testing tic suppression: comparing the effects of dexmethylphenidate to no medication in children and adolescents with attention-deficit/hyperactivity disorder and Tourette’s disorder. J Child Adolesc Psychopharmacol. 2010; 20(4): 283–9. PubMed Abstract | Publisher Full Text | Free Full Text
5. Conelea CA, Wellen B, Woods DW, et al.: Patterns and predictors of tic suppressibility in youth with tic disorders. bioRxiv. 2017; 204933. Publisher Full Text
6. Black KJ, Black ER, Greene DJ, et al.: Provisional Tic Disorder: What to tell parents when their child first starts ticcing [version 1; referees: 3 approved]. F1000Res. 2016; 5: 696. PubMed Abstract | Publisher Full Text | Free Full Text
7. Greene DJ, Koller JM, Robichaux-Viehoever A, et al.: Reward enhances tic suppression in children within months of tic disorder onset. Dev Cogn Neurosci. 2015; 11: 65–74. PubMed Abstract | Publisher Full Text | Free Full Text
8. American Psychiatric Association: Obsessive-Compulsive and Related Disorders. In: Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition. Arlington, VA: American Psychiatric Association; 2013. Publisher Full Text
9. Black JK, Koller JM, Black KJ: TicTimer software for measuring tic suppression. Authorea, 2017. Publisher Full Text
10. Black JK, Black KJ: Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Res. 2017; 6: 2150. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 24 Aug 2017

Author details Author details

Jonathan K. Black
Roles: Methodology, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan M. Koller
Roles: Methodology, Software, Writing – Review & Editing

Kevin J. Black
Roles: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Software development and manuscript preparation were funded in part by the U.S. National Institutes of Health (NIH), grant numbers K24 MH087913, R21 NS091635, and R01 MH104030.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 22 Dec 2017, 6:1560

https://doi.org/10.12688/f1000research.12327.2

version 1

Published: 24 Aug 2017, 6:1560

https://doi.org/10.12688/f1000research.12327.1

© 2017 Black JK et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Black JK, Koller JM and Black KJ. TicTimer software for measuring tic suppression [version 2; peer review: 2 approved] F1000Research 2017, 6:1560 (https://doi.org/10.12688/f1000research.12327.2)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 22 Dec 2017

Revised

Views

Reviewer Report 12 Jan 2018

Danielle C. Cath, Department of Psychiatry, University Medical Centre Groningen, Groningen, The Netherlands

Approved

https://doi.org/10.5256/f1000research.14702.r29268

I went through the revision and responses by the authors to my ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 24 Aug 2017

Views

Reviewer Report 06 Nov 2017

Patrick Haggard, Institute of Cognitive Neuroscience & Department of Psychology, University College London, London, WC1N 3AR, UK

Approved

https://doi.org/10.5256/f1000research.13345.r26009

This is a simple but functional piece of software for logging tics, and automating a reward schedule based on them.

The hardware/software combination will be useful for researchers and possibly for behavioural analysis/clinical training – though this publication does not aim to describe clinical outcomes.

The software does not appear to actually detect tics automatically – there is no interface to accelometry for example. Rather an examiner seems to have to observe the examinee constantly, and enter manually when a tic occurs, presumably by pressing a button. However, this requirement is unfortunately not specified directly in the text. It also means there is a subjective component: how does the examiner judge what is a tic and what is, for example, a voluntary action: these are tricky questions which face researchers in this area.

The data is logged in a simple way that is clearly explained, and suitable for postprocessing using standard packages – but the user seems to have to write their own scripts to produce synthetic reports.

The package logs occurrence of a 10 s period in which no tic is reported – but this time window appears arbitrary: it would be useful (a) to justify it, and (b) to allow the user to change it, either at run time, or in subsequent sensitivity analysis of the log files: how long a period of tic-free behaviour should count as a success event?

The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children.

Is the rationale for developing the new software tool clearly explained?

Yes
Is the description of the software tool technically sound?

Partly
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Neurocognition of human sensorimotor function

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

22 Dec 2017

Author Response F1000Research Advisory Board Member
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.
- The software doesn't
... Continue reading
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.

The software doesn't detect tics automatically; the accuracy of any rater-identified tics depend on the skill of the rater. These points are all valid. The scope of the software is limited, primarily being useful to track and time the tics as noted by an expert observer, who indeed logs each tic by pressing a keyboard button or clicking a mouse. We have updated the text to clarify these details.

"The user seems to have to write their own scripts to produce synthetic reports." We do supply (via GitHub) the python script TT_Data.py for summarizing each primary data file, but apparently we failed to provide a direct link to this script. We have corrected this omission in this revision.

The choice of 10 seconds as the tic-free interval of interest is arbitrary. We chose the 10s period because of its prior use in the studies cited, but you’re correct that this choice is otherwise arbitrary. We have explained this choice in the revised ms. The suggestions in (b) are interesting, but to some extent outside of our plan for the software. We have modified the source code, however, to make a change in tic-free intervals easier to make at compile time.

"The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children." Yes, we also see that as one of its key advantages. We expect TicTimer could be applied equally well to studies of adults.
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.

The software doesn't detect tics automatically; the accuracy of any rater-identified tics depend on the skill of the rater. These points are all valid. The scope of the software is limited, primarily being useful to track and time the tics as noted by an expert observer, who indeed logs each tic by pressing a keyboard button or clicking a mouse. We have updated the text to clarify these details.

"The user seems to have to write their own scripts to produce synthetic reports." We do supply (via GitHub) the python script TT_Data.py for summarizing each primary data file, but apparently we failed to provide a direct link to this script. We have corrected this omission in this revision.

The choice of 10 seconds as the tic-free interval of interest is arbitrary. We chose the 10s period because of its prior use in the studies cited, but you’re correct that this choice is otherwise arbitrary. We have explained this choice in the revised ms. The suggestions in (b) are interesting, but to some extent outside of our plan for the software. We have modified the source code, however, to make a change in tic-free intervals easier to make at compile time.

"The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children." Yes, we also see that as one of its key advantages. We expect TicTimer could be applied equally well to studies of adults.
Competing Interests: none Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

22 Dec 2017

Author Response F1000Research Advisory Board Member
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.
- The software doesn't
... Continue reading
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.

The software doesn't detect tics automatically; the accuracy of any rater-identified tics depend on the skill of the rater. These points are all valid. The scope of the software is limited, primarily being useful to track and time the tics as noted by an expert observer, who indeed logs each tic by pressing a keyboard button or clicking a mouse. We have updated the text to clarify these details.

"The user seems to have to write their own scripts to produce synthetic reports." We do supply (via GitHub) the python script TT_Data.py for summarizing each primary data file, but apparently we failed to provide a direct link to this script. We have corrected this omission in this revision.

The choice of 10 seconds as the tic-free interval of interest is arbitrary. We chose the 10s period because of its prior use in the studies cited, but you’re correct that this choice is otherwise arbitrary. We have explained this choice in the revised ms. The suggestions in (b) are interesting, but to some extent outside of our plan for the software. We have modified the source code, however, to make a change in tic-free intervals easier to make at compile time.

"The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children." Yes, we also see that as one of its key advantages. We expect TicTimer could be applied equally well to studies of adults.
We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.

The software doesn't detect tics automatically; the accuracy of any rater-identified tics depend on the skill of the rater. These points are all valid. The scope of the software is limited, primarily being useful to track and time the tics as noted by an expert observer, who indeed logs each tic by pressing a keyboard button or clicking a mouse. We have updated the text to clarify these details.

"The user seems to have to write their own scripts to produce synthetic reports." We do supply (via GitHub) the python script TT_Data.py for summarizing each primary data file, but apparently we failed to provide a direct link to this script. We have corrected this omission in this revision.

The choice of 10 seconds as the tic-free interval of interest is arbitrary. We chose the 10s period because of its prior use in the studies cited, but you’re correct that this choice is otherwise arbitrary. We have explained this choice in the revised ms. The suggestions in (b) are interesting, but to some extent outside of our plan for the software. We have modified the source code, however, to make a change in tic-free intervals easier to make at compile time.

"The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children." Yes, we also see that as one of its key advantages. We expect TicTimer could be applied equally well to studies of adults.
Competing Interests: none Close
Report a concern

Views

Reviewer Report 30 Oct 2017

Danielle C. Cath, Department of Psychiatry, University Medical Centre Groningen, Groningen, The Netherlands

Approved with Reservations

https://doi.org/10.5256/f1000research.13345.r27424

This tool might be very useful in clinical practice to make tic suppression more objectively measured, and to substantiate the direct results of tic suppression practicing. It is potentially a useful addition to current practice of CBT in tic suppression, ... Continue reading

Direct comparison with other potential ways to generate objective quantifiable results; for instance the advantages compared to video-based tic counting.
I miss reflection on potential disadvantages of the tool, and limitations of the tool (for instance in practicing in daily life, not only in the lab, is unclear.
In the plain text output, I miss summary information across sessions, and f.i. graphs on tic course across sessions that could be presented to the patient and discussed. The latter would be really helpful for patients and therapist
The device can in no way be used in more naturalistic sessions outside the treatment room, the authors should speculate about this, and about future developments to make the tool more practical and applicable in different situations and contexts.
I miss pilot information on direct comparisons between conventional tic counting and the counting with the aid of the tictimer.

Is the rationale for developing the new software tool clearly explained?

Partly
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Partly
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Science practioner, specialised in research and clinics of diagnosis and treatment of tics and Tourettes Disorder

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

22 Dec 2017

Author Response F1000Research Advisory Board Member
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic ... Continue reading
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic Suppression Protocol in a clinical research office setting. This fact affects many of the specific comments below. We have revised the Introduction to address this point.

Here are responses to each item in your review:

comparing to other tic counting methods: This tool can be used (and we have used it) to facilitate counting tics on video recordings. The main drawback with using the software to count and time tics on video recordings is that one cannot give “live” feedback in the Differential Reinforcement of Other (zero-rate ticcing) condition.

limitations: We have rewritten the introduction to clarify the limited intended purpose, and clarify in Discussion that use outside of the research office will require adaptations.

graphs showing changes over time, for clinical use: This is a useful feature that may be added in the future. In the meantime, we point out that the information in Supplementary File 9 can be imported easily into a spreadsheet program for creating such graphs.

can't be used in more naturalistic settings: Correct. See our initial comment, above. We have in fact adapted some features of TicTimer for a web-based tool intended for more naturalistic (home-based) tic suppression timing.^[1]

comparison of TicTimer to conventional tic counting: We do not feel this is necessary, because (as we apparently failed to explain adequately) in each case an expert human is watching the person with tics. TicTimer adds the (more or less) precise timing of each tic to the record.

[1] Black JK and Black KJ. Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Research 2017, 6:2150 (doi: 10.12688/f1000research.13460.1)
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic Suppression Protocol in a clinical research office setting. This fact affects many of the specific comments below. We have revised the Introduction to address this point.

Here are responses to each item in your review:

comparing to other tic counting methods: This tool can be used (and we have used it) to facilitate counting tics on video recordings. The main drawback with using the software to count and time tics on video recordings is that one cannot give “live” feedback in the Differential Reinforcement of Other (zero-rate ticcing) condition.

limitations: We have rewritten the introduction to clarify the limited intended purpose, and clarify in Discussion that use outside of the research office will require adaptations.

graphs showing changes over time, for clinical use: This is a useful feature that may be added in the future. In the meantime, we point out that the information in Supplementary File 9 can be imported easily into a spreadsheet program for creating such graphs.

can't be used in more naturalistic settings: Correct. See our initial comment, above. We have in fact adapted some features of TicTimer for a web-based tool intended for more naturalistic (home-based) tic suppression timing.^[1]

comparison of TicTimer to conventional tic counting: We do not feel this is necessary, because (as we apparently failed to explain adequately) in each case an expert human is watching the person with tics. TicTimer adds the (more or less) precise timing of each tic to the record.

[1] Black JK and Black KJ. Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Research 2017, 6:2150 (doi: 10.12688/f1000research.13460.1)
Competing Interests: none Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

22 Dec 2017

Author Response F1000Research Advisory Board Member
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic ... Continue reading
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic Suppression Protocol in a clinical research office setting. This fact affects many of the specific comments below. We have revised the Introduction to address this point.

Here are responses to each item in your review:

comparing to other tic counting methods: This tool can be used (and we have used it) to facilitate counting tics on video recordings. The main drawback with using the software to count and time tics on video recordings is that one cannot give “live” feedback in the Differential Reinforcement of Other (zero-rate ticcing) condition.

limitations: We have rewritten the introduction to clarify the limited intended purpose, and clarify in Discussion that use outside of the research office will require adaptations.

graphs showing changes over time, for clinical use: This is a useful feature that may be added in the future. In the meantime, we point out that the information in Supplementary File 9 can be imported easily into a spreadsheet program for creating such graphs.

can't be used in more naturalistic settings: Correct. See our initial comment, above. We have in fact adapted some features of TicTimer for a web-based tool intended for more naturalistic (home-based) tic suppression timing.^[1]

comparison of TicTimer to conventional tic counting: We do not feel this is necessary, because (as we apparently failed to explain adequately) in each case an expert human is watching the person with tics. TicTimer adds the (more or less) precise timing of each tic to the record.

[1] Black JK and Black KJ. Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Research 2017, 6:2150 (doi: 10.12688/f1000research.13460.1)
We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic Suppression Protocol in a clinical research office setting. This fact affects many of the specific comments below. We have revised the Introduction to address this point.

Here are responses to each item in your review:

comparing to other tic counting methods: This tool can be used (and we have used it) to facilitate counting tics on video recordings. The main drawback with using the software to count and time tics on video recordings is that one cannot give “live” feedback in the Differential Reinforcement of Other (zero-rate ticcing) condition.

limitations: We have rewritten the introduction to clarify the limited intended purpose, and clarify in Discussion that use outside of the research office will require adaptations.

graphs showing changes over time, for clinical use: This is a useful feature that may be added in the future. In the meantime, we point out that the information in Supplementary File 9 can be imported easily into a spreadsheet program for creating such graphs.

can't be used in more naturalistic settings: Correct. See our initial comment, above. We have in fact adapted some features of TicTimer for a web-based tool intended for more naturalistic (home-based) tic suppression timing.^[1]

comparison of TicTimer to conventional tic counting: We do not feel this is necessary, because (as we apparently failed to explain adequately) in each case an expert human is watching the person with tics. TicTimer adds the (more or less) precise timing of each tic to the record.

[1] Black JK and Black KJ. Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Research 2017, 6:2150 (doi: 10.12688/f1000research.13460.1)
Competing Interests: none Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 24 Aug 2017

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 22 Dec 17	read
Version 1 24 Aug 17	read	read

Danielle C. Cath, University Medical Centre Groningen, Groningen, The Netherlands
Patrick Haggard, University College London, London, UK

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

4 Views

12 Jan 2018 | for Version 2

Danielle C. Cath, Department of Psychiatry, University Medical Centre Groningen, Groningen, The Netherlands

4 Views Cite this report Responses(0)

Approved

I went through the revision and responses by the authors to my comments, and I have no further worries or comments, this work is worthwhile indexing.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

8 Views

06 Nov 2017 | for Version 1

Patrick Haggard, Institute of Cognitive Neuroscience & Department of Psychology, University College London, London, WC1N 3AR, UK

8 Views Cite this report Responses(1)

Approved

Is the rationale for developing the new software tool clearly explained?

Yes
Is the description of the software tool technically sound?

Partly
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Neurocognition of human sensorimotor function

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response F1000Research Advisory Board Member

22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

We are grateful for the careful attention and thoughtful suggestions of Prof. Haggard. Here we reply to each point in his review of version 1.

The software doesn't detect tics automatically; the accuracy of any rater-identified tics depend on the skill of the rater. These points are all valid. The scope of the software is limited, primarily being useful to track and time the tics as noted by an expert observer, who indeed logs each tic by pressing a keyboard button or clicking a mouse. We have updated the text to clarify these details.
"The user seems to have to write their own scripts to produce synthetic reports." We do supply (via GitHub) the python script TT_Data.py for summarizing each primary data file, but apparently we failed to provide a direct link to this script. We have corrected this omission in this revision.
The choice of 10 seconds as the tic-free interval of interest is arbitrary. We chose the 10s period because of its prior use in the studies cited, but you’re correct that this choice is otherwise arbitrary. We have explained this choice in the revised ms. The suggestions in (b) are interesting, but to some extent outside of our plan for the software. We have modified the source code, however, to make a change in tic-free intervals easier to make at compile time.
"The strength of the tool seems to be the automation of reward scheduling for studies of voluntary tic suppression in children." Yes, we also see that as one of its key advantages. We expect TicTimer could be applied equally well to studies of adults.

View more View less

Competing Interests

none

Back to all reports

Reviewer Report

9 Views

30 Oct 2017 | for Version 1

Danielle C. Cath, Department of Psychiatry, University Medical Centre Groningen, Groningen, The Netherlands

9 Views Cite this report Responses(1)

Approved With Reservations

Direct comparison with other potential ways to generate objective quantifiable results; for instance the advantages compared to video-based tic counting.
I miss reflection on potential disadvantages of the tool, and limitations of the tool (for instance in practicing in daily life, not only in the lab, is unclear.
In the plain text output, I miss summary information across sessions, and f.i. graphs on tic course across sessions that could be presented to the patient and discussed. The latter would be really helpful for patients and therapist
The device can in no way be used in more naturalistic sessions outside the treatment room, the authors should speculate about this, and about future developments to make the tool more practical and applicable in different situations and contexts.
I miss pilot information on direct comparisons between conventional tic counting and the counting with the aid of the tictimer.

Is the rationale for developing the new software tool clearly explained?

Partly
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Partly
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Science practioner, specialised in research and clinics of diagnosis and treatment of tics and Tourettes Disorder

Respond to this report

Responses (1)

Author Response F1000Research Advisory Board Member

22 Dec 2017

Kevin J Black, Department of Neuroscience, Washington University School of Medicine, St. Louis, 63110, USA

We thank Prof. Cath for her thoughtful comments. We failed to clarify adequately the primary purpose of this software tool. It was designed rather narrowly for use in a standardized Tic Suppression Protocol in a clinical research office setting. This fact affects many of the specific comments below. We have revised the Introduction to address this point.

Here are responses to each item in your review:

comparing to other tic counting methods: This tool can be used (and we have used it) to facilitate counting tics on video recordings. The main drawback with using the software to count and time tics on video recordings is that one cannot give “live” feedback in the Differential Reinforcement of Other (zero-rate ticcing) condition.
limitations: We have rewritten the introduction to clarify the limited intended purpose, and clarify in Discussion that use outside of the research office will require adaptations.
graphs showing changes over time, for clinical use: This is a useful feature that may be added in the future. In the meantime, we point out that the information in Supplementary File 9 can be imported easily into a spreadsheet program for creating such graphs.
can't be used in more naturalistic settings: Correct. See our initial comment, above. We have in fact adapted some features of TicTimer for a web-based tool intended for more naturalistic (home-based) tic suppression timing.^[1]
comparison of TicTimer to conventional tic counting: We do not feel this is necessary, because (as we apparently failed to explain adequately) in each case an expert human is watching the person with tics. TicTimer adds the (more or less) precise timing of each tic to the record.

[1] Black JK and Black KJ. Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Research 2017, 6:2150 (doi: 10.12688/f1000research.13460.1)

View more View less

Competing Interests

none

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Woods DW, Himle MB: Creating tic suppression: comparing the effects of verbal instruction to differential reinforcement. J Appl Behav Anal. 2004; 37(3): 417–20. PubMed Abstract | Free Full Text

[2] 2. Himle MB, Woods DW: An experimental evaluation of tic suppression and the tic rebound effect. Behav Res Ther. 2005; 43(11): 1443–51. PubMed Abstract | Publisher Full Text

[3] 3. Himle MB, Woods DW, Bunaciu L: Evaluating the role of contingency in differentially reinforced tic suppression. J Appl Behav Anal. 2008; 41(2): 285–9. PubMed Abstract | Free Full Text

[4] 4. Lyon GJ, Samar SM, Conelea C, et al.: Testing tic suppression: comparing the effects of dexmethylphenidate to no medication in children and adolescents with attention-deficit/hyperactivity disorder and Tourette’s disorder. J Child Adolesc Psychopharmacol. 2010; 20(4): 283–9. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Conelea CA, Wellen B, Woods DW, et al.: Patterns and predictors of tic suppressibility in youth with tic disorders. bioRxiv. 2017; 204933. Publisher Full Text

[6] 6. Black KJ, Black ER, Greene DJ, et al.: Provisional Tic Disorder: What to tell parents when their child first starts ticcing [version 1; referees: 3 approved]. F1000Res. 2016; 5: 696. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Greene DJ, Koller JM, Robichaux-Viehoever A, et al.: Reward enhances tic suppression in children within months of tic disorder onset. Dev Cogn Neurosci. 2015; 11: 65–74. PubMed Abstract | Publisher Full Text | Free Full Text

[8] 8. American Psychiatric Association: Obsessive-Compulsive and Related Disorders. In: Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition. Arlington, VA: American Psychiatric Association; 2013. Publisher Full Text

[9] 9. Black JK, Koller JM, Black KJ: TicTimer software for measuring tic suppression. Authorea, 2017. Publisher Full Text

[10] 10. Black JK, Black KJ: Software for web-based tic suppression training [version 1; referees: awaiting peer review]. F1000Res. 2017; 6: 2150. Publisher Full Text

TicTimer software for measuring tic suppression

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Implementation

Figure 1. From left to right: Token dispenser box, Student Trainer Interface with external pushbutton, box containing relay module, connected via USB cable to laptop running TicTrainer.

Operation

Use cases

Conclusions

Software availability

Competing interests

Grant information

Acknowledgments

Supplementary material

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated