Conversations about Sleep: Clinical Data Wrangling

class: center, middle, inverse, title-slide

# Conversations about Sleep: Clinical Data Wrangling
### Ted Laderas, Nicole Weiskopf and Eilis Boudreau<br><span style="font-size: 65%;">Medical Informatics and Clinical Epidemiology<br>Oregon Health & Science University</span>
### AMIA Informatics Educators Forum 2019<br><span style="font-size: 65%;">S15: Interactive Demonstrations - Learning Environments to Support<br> Biomedical Informatics, Data Science, and Clinical Data Quality</span><br><i class="fas  fa-link "></i> <a href="http://bit.ly/ted-ief2019">bit.ly/ted-ief2019</a><br><i class="fab  fa-twitter "></i> <a href="https://twitter.com/tladeras">tladeras</a>

---

layout: true
background-image: url(image/ohsu.png), url(image/logo.png)
background-position: 95% 5%, 95% 95%
background-size: 7%, 20%

---
# Disclosure

No disclosures to report.

---
# Learning Objectives for this Session

- **Learn** about the challenges of developing multidisciplinary workshops
- **Understand** why exploratory data analysis is an important data science skill
- **Discuss** why iterative learning is helpful in understanding clinical data

---
# Introduction: Ted Laderas

.pull-left[ 
- Assistant Professor, Oregon Health & Science University
  - Department of Medical Informatics and Clinical Epidemiology
  - Division of Bioinformatics and Computational Biomedicine 
- Research Interests
  - **Education (active learning)**
  - **Interactive Visualization**
  - **Open Science**
  - Immunoinformatics
  - Drug Sensitivity/Resistance in Cancer
  ]

.pull-right[
<img src = "image/LaderasTed-small.jpg" height = 300>
]
---
# Overview

- Why clinical data wrangling?
- Why the workshop?
- What is the workshop?
- Components of the workshop
- Extensions
- Availability

---
# Motivation: Why?

![https://twitter.com/jd_wilko/status/1138899593826131973](image/wilkinson.png)

---

# Applies to Informaticians, Too

![](image/wilkinson-revised.png)

- Emphasize successful informatics being collaboration
- Research is multi-disciplinary
- Need for Translational Research Workforce
- Bridge gaps between computational biology and clinical realms

---
# Luckily, there was funding

- NLM T15 Training Grant Supplement for Data Science
- Allowed a diverse team to come together:
    - Eilis Boudreau, MD/PhD (Sleep Neurologist)
    - Nicole Weiskopf, PhD (EHR Data Quality)
    - Ted Laderas, PhD (Education, Interactive Visualization)

---
# Clinical Data Wrangling

.pull-left[
- Workshop for incoming graduate students
- Audience: Clinical/Bioinformatics Students
  - Assumes no expertise with R/RStudio
- Introduce students to data quality issues with clinical data
- Context: Secondary use of a clinical dataset
  - Can we use it for prediction of cardiovascular disease?
  - What variables can we use?
]

.pull-right[
<img src = "image/data-wrangling.png" height = 400>
]

---

# Learning Objectives

.pull-left[
1. **Understand** the biology of sleep and sleep apnea and how the biology informs the covariates measured in the Sleep Heart Health Study
1. **Understand** the usefulness and challenges with working with clinical data
1. **Learn** Exploratory Data Analysis techniques and use them to **inform/analyse** model building
1. **Build** and **Evaluate** a predictive model (logistic regression) using simple diagnostics.
1. **Communicate** your findings in a team setting.
]
.pull-right[
<img src = "image/data-wrangling.png" height = 400>
]
---

# Challenges: Where to get data?

- Partnership with National Sleep Research Resource 
    - http://sleepdata.org
- Sleep Heart Health Study data 
    - ~5700 patients, 1200 covariates
- Requires students to sign [data use agreement]()

---
## Sleep Covariates

#hcxuuaqiyu .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #000000;
  font-size: 16px;
  background-color: #FFFFFF;
  /* table.background.color */
  width: auto;
  /* table.width */
  border-top-style: solid;
  /* table.border.top.style */
  border-top-width: 2px;
  /* table.border.top.width */
  border-top-color: #A8A8A8;
  /* table.border.top.color */
}

#hcxuuaqiyu .gt_heading {
  background-color: #FFFFFF;
  /* heading.background.color */
  border-bottom-color: #FFFFFF;
}

#hcxuuaqiyu .gt_title {
  color: #000000;
  font-size: 125%;
  /* heading.title.font.size */
  padding-top: 4px;
  /* heading.top.padding */
  padding-bottom: 1px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#hcxuuaqiyu .gt_subtitle {
  color: #000000;
  font-size: 85%;
  /* heading.subtitle.font.size */
  padding-top: 1px;
  padding-bottom: 4px;
  /* heading.bottom.padding */
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#hcxuuaqiyu .gt_bottom_border {
  border-bottom-style: solid;
  /* heading.border.bottom.style */
  border-bottom-width: 2px;
  /* heading.border.bottom.width */
  border-bottom-color: #A8A8A8;
  /* heading.border.bottom.color */
}

#hcxuuaqiyu .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  padding-top: 4px;
  padding-bottom: 4px;
}

#hcxuuaqiyu .gt_col_heading {
  color: #000000;
  background-color: #FFFFFF;
  /* column_labels.background.color */
  font-size: 16px;
  /* column_labels.font.size */
  font-weight: initial;
  /* column_labels.font.weight */
  vertical-align: middle;
  padding: 10px;
  margin: 10px;
}

#hcxuuaqiyu .gt_sep_right {
  border-right: 5px solid #FFFFFF;
}

#hcxuuaqiyu .gt_group_heading {
  padding: 8px;
  color: #000000;
  background-color: #FFFFFF;
  /* row_group.background.color */
  font-size: 16px;
  /* row_group.font.size */
  font-weight: initial;
  /* row_group.font.weight */
  border-top-style: solid;
  /* row_group.border.top.style */
  border-top-width: 2px;
  /* row_group.border.top.width */
  border-top-color: #A8A8A8;
  /* row_group.border.top.color */
  border-bottom-style: solid;
  /* row_group.border.bottom.style */
  border-bottom-width: 2px;
  /* row_group.border.bottom.width */
  border-bottom-color: #A8A8A8;
  /* row_group.border.bottom.color */
  vertical-align: middle;
}

#hcxuuaqiyu .gt_empty_group_heading {
  padding: 0.5px;
  color: #000000;
  background-color: #FFFFFF;
  /* row_group.background.color */
  font-size: 16px;
  /* row_group.font.size */
  font-weight: initial;
  /* row_group.font.weight */
  border-top-style: solid;
  /* row_group.border.top.style */
  border-top-width: 2px;
  /* row_group.border.top.width */
  border-top-color: #A8A8A8;
  /* row_group.border.top.color */
  border-bottom-style: solid;
  /* row_group.border.bottom.style */
  border-bottom-width: 2px;
  /* row_group.border.bottom.width */
  border-bottom-color: #A8A8A8;
  /* row_group.border.bottom.color */
  vertical-align: middle;
}

#hcxuuaqiyu .gt_striped {
  background-color: #f2f2f2;
}

#hcxuuaqiyu .gt_from_md > :first-child {
  margin-top: 0;
}

#hcxuuaqiyu .gt_from_md > :last-child {
  margin-bottom: 0;
}

#hcxuuaqiyu .gt_row {
  padding: 8px;
  /* row.padding */
  margin: 10px;
  vertical-align: middle;
}

#hcxuuaqiyu .gt_stub {
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #A8A8A8;
  padding-left: 12px;
}

#hcxuuaqiyu .gt_summary_row {
  color: #000000;
  background-color: #FFFFFF;
  /* summary_row.background.color */
  padding: 8px;
  /* summary_row.padding */
  text-transform: inherit;
  /* summary_row.text_transform */
}

#hcxuuaqiyu .gt_grand_summary_row {
  color: #000000;
  background-color: #FFFFFF;
  /* grand_summary_row.background.color */
  padding: 8px;
  /* grand_summary_row.padding */
  text-transform: inherit;
  /* grand_summary_row.text_transform */
}

#hcxuuaqiyu .gt_first_summary_row {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
}

#hcxuuaqiyu .gt_first_grand_summary_row {
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #A8A8A8;
}

#hcxuuaqiyu .gt_table_body {
  border-top-style: solid;
  /* table_body.border.top.style */
  border-top-width: 2px;
  /* table_body.border.top.width */
  border-top-color: #A8A8A8;
  /* table_body.border.top.color */
  border-bottom-style: solid;
  /* table_body.border.bottom.style */
  border-bottom-width: 2px;
  /* table_body.border.bottom.width */
  border-bottom-color: #A8A8A8;
  /* table_body.border.bottom.color */
}

#hcxuuaqiyu .gt_footnote {
  font-size: 90%;
  /* footnote.font.size */
  padding: 4px;
  /* footnote.padding */
}

#hcxuuaqiyu .gt_sourcenote {
  font-size: 90%;
  /* sourcenote.font.size */
  padding: 4px;
  /* sourcenote.padding */
}

#hcxuuaqiyu .gt_center {
  text-align: center;
}

#hcxuuaqiyu .gt_left {
  text-align: left;
}

#hcxuuaqiyu .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#hcxuuaqiyu .gt_font_normal {
  font-weight: normal;
}

#hcxuuaqiyu .gt_font_bold {
  font-weight: bold;
}

#hcxuuaqiyu .gt_font_italic {
  font-style: italic;
}

#hcxuuaqiyu .gt_super {
  font-size: 65%;
}

#hcxuuaqiyu .gt_footnote_glyph {
  font-style: italic;
  font-size: 65%;
}
</style>
<div id="hcxuuaqiyu" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<table class='gt_table'>
<tr>
<th class='gt_col_heading gt_center' rowspan='1' colspan='1'>Covariate</th>
<th class='gt_col_heading gt_center' rowspan='1' colspan='1'>Description</th>
</tr>
<tbody class='gt_table_body'>
<tr>
<td class='gt_row gt_center'>any_cvd</td>
<td class='gt_row gt_center'>Any Cardiovascular Disease (CVD) Since Baseline?</td>
</tr>
<tr>
<td class='gt_row gt_center gt_striped'>age_s1</td>
<td class='gt_row gt_center gt_striped'>Age at Sleep Heart Health Study Visit One (SHHS1)</td>
</tr>
<tr>
<td class='gt_row gt_center'>gender</td>
<td class='gt_row gt_center'>Gender</td>
</tr>
<tr>
<td class='gt_row gt_center gt_striped'>race</td>
<td class='gt_row gt_center gt_striped'>Race</td>
</tr>
<tr>
<td class='gt_row gt_center'>bmi_s1</td>
<td class='gt_row gt_center'>Body mass index (BMI) (Sleep Heart Health Study Visit One (SHHS1))</td>
</tr>
<tr>
<td class='gt_row gt_center gt_striped'>neck20</td>
<td class='gt_row gt_center gt_striped'>Neck Circumference (Sleep Heart Health Study Visit One (SHHS1))</td>
</tr>
<tr>
<td class='gt_row gt_center'>systbp</td>
<td class='gt_row gt_center'>Average Systolic blood pressure (BP) (Sleep Heart Health Study Visit One (SHHS1))</td>
</tr>
<tr>
<td class='gt_row gt_center gt_striped'>chol</td>
<td class='gt_row gt_center gt_striped'>Cholesterol</td>
</tr>
<tr>
<td class='gt_row gt_center'>hdl</td>
<td class='gt_row gt_center'>High-density lipoprotein (HDL) cholesterol</td>
</tr>
<tr>
<td class='gt_row gt_center gt_striped'>ahi_a0h3</td>
<td class='gt_row gt_center gt_striped'>Apnea-Hypopnea Index (AHI) >= 3% - number of [all apneas] and [hypopneas with >= 3% oxygen desaturation] per hour of sleep</td>
</tr>
</tbody>
</table>

</div>
---
# Workshop is split over multiple days

.pull-left[
- **Day 1** (8 Hrs):
  - Logistics
  - Intro to sleep data
  - Intro to data reuse and data quality
  - Data exploration of sleep data
  - Intro to logistic regression
  ]
.pull-right[
- **Day 2** (2 hrs): 
  - Logistic regression Q&A
  - Assignment: Is Race appropriate?
- **Day 3** (2 hrs):
  - Is hypertension appropriate?
  - Final presentation prep
- **Day 4** (1.5 hrs):
  - Final Presentation by groups
]

---
# Social Learning

- Code of Conduct
- Introduce students to each other
- Psychological Safety
- Diverse/Heterogeneous teams

.footnote[[What Every Data Scientist Should Know about Education](https://resources.rstudio.com/webinars/what-every-data-scientist-should-know-about-education-greg-wilson)]

---

# Lecture: Background about Sleep

- Eilis Boudreau: Sleep Neurologist
- Role of sleep disorders in cardiovascular disease 
- How we measure sleep disorders

.footnote[[Image from Wikipedia](https://en.wikipedia.org/wiki/Sleep_apnea#/media/File:Obstruction_ventilation_apn%C3%A9e_sommeil.svg)]

---

# Lecture: Data Reuse and Data Quality Issues

- Nicole Weiskopf
- Expose students early on to data quality issues
- Hazards and bonuses of data reuse
- Biases of Clinical Data

<img src="image/healthcare-data.png" height = 350>
---

# Making Decisions by Exploring Data

.pull-left[
- Shiny app (burro) that lets students explore dataset interactively
- Code-less, assumes no coding expertise
- Can be deployed as a website
]
.pull-right[
![](image/burro2.png)
]

---
# burro: Visual Summary of the Data

.footnote[[visdat package by Nick Tierney](http://visdat.njtierney.com/)]
---

# burro: Single Categories to CrossTabs

---

# burro: Histograms to Boxplots

---
# Try burro out!

http://bit.ly/csv_burro

---

# Building Predictive Models by Making Decisions

- Rstudio notebook that [guides students through modeling process](https://laderast.github.io/clinical_data_wrangling/notebooks/predictive_model.nb.html)
  - Choice of covariates
  - Simple data cleaning/impact on dataset
  
![](image/nas.png)

---

# Modeling the Data

.pull-left[
- Assessing Impact of logistic regression
- Understanding and evaluating model predictions
- Splitting Data: Test/Train Datasets
- Diagnostic Testing and their uses
  -Patients who are on the edge
    ]
    
.pull-right[ 
1. **patient 1** is older 65, Male, and has a higher BMI (44). 
1. **patient2** is younger 25, Female and has a lower BMI (23). 
1. **patient3** is a middle aged (42), Female patient with a middling BMI (31).
]

---

# Day 2/3: Iterating your model: what about Race/Hypertension?

- Understanding the contributions of `race`
  - How is it defined? 
  - How does our study population compare to the US?
- Understanding `hypertension` as a variable
  - Medication Lists
  - Blood pressure measurements
---

# Day 4: Final Analysis: Making and Documenting Critical Decisions

.pull-left[- Final presentation is an R Notebook
- What variables did you use at each step?
- Why did you include them?
- What were your values?
- How did that affect your results?]

.pull-right[<img src="image/clinical_wrangling.png" height = 300>]

---

# Challenges: There is no right answer

- Data Use Agreements required for student participation
- Students wanted to choose an "optimal" threshold for models
- Challenges in understanding importance of how a clinical test would be used

---

# Outcomes

- Sets the tone for our program
- Good opportunity for clinical/bioinformatics students to meet
- Currently assessing impact on students after 1 year in program

---

# Extensions: Data Science for MD/PhDs

- Reframe as clinical case (Kristen Stevens)

---

# Extensions: Data Scavenger Hunts

.pull-left[ 
- [Exploring the NHANES dataset](http://bit.ly/csv_burro)
- [OHSU BioData Club activity](http://biodata-club.github.io)
]
.pull-right[![](image/Hex-BDP-small.png)]
---

# Acknowledgements:

- Bill Hersh
- Diane Doctor
- T15 NLM Training Supplement Grant: 5T15LM007088
- Susan Redline/Dan Mobley at NSRR
- Incoming class of 2018 informatics students
- MD/PhD Students
- BioData Club Members

---

# Are you interested?

Keep in touch! We want others to utilize the workshop!

laderast@ohsu.edu | @tladeras | http://laderast.github.io

- These slides: http://bit.ly/ted-ief2019
- Burro: http://laderast.github.io/burro
- Workshop: http://laderast.github.io/clinical_data_wrangling/