Nearly every quiz question should have an `answer` box after it to explain why the correct answer is correct. This text will show up after the learner answers the question correctly or clicks to have the right answer revealed. It can be as long as you like, and allows any markdown formatting (you can embed pictures or videos, links, etc.). Use `

` to mark these sections with special styling, so that they're visually distinct from the rest of the quiz. The style for `"answer"` is defined in the css file. For this context to show up automatically when the learner answers the question correctly or clicks to have the right answer revealed, it needs to be surrounded by `***` (at least three, but you can use more if you want a more visually distinct horizontal marker in your md file).

*** ``` ### Multiple choice Here is the first question. It's multiple choice. [(X)] This answer is right [( )] This is wrong [( )] Also wrong [[?]] Hint: Provide a hint here if you like. Hints are marked with the ? [[?]] Hint: You can include as many hints as you want. ***

*** You can have questions with multiple correct answers. ``` Select all of the following correct choices: [[ ]] Not this one [[X]] This is one of the correct ones [[X]] Here's another correct one [[ ]] This one is wrong, though [[?]] Hint: Remember to select ALL of the correct choices. ***

Here is the answer box for this question.

*** ``` Select all of the following correct choices: [[ ]] Not this one [[X]] This is one of the correct ones [[X]] Here's another correct one [[ ]] This one is wrong, though [[?]] Hint: Remember to select ALL of the correct choices. ***

Here is the answer box for this question.

*** ``` True or False: This statement is NOT true. ;) [( )] TRUE [(X)] FALSE ***

Here is the answer box for this question.

*** ``` True or False: This statement is NOT true. ;) [( )] TRUE [(X)] FALSE ***

Here is the answer box for this question.

*** ### Short answer/text response Note that, without any additional script, to get it marked "correct" the learner has to enter it exactly as you do. We can allow some flexibility in what we accept as correct answers for text by adding a little script after the answer, though. For the following, either "right answer" or "correct answer" (not case sensitive) will be accepted: ``` What is the right answer? [[right answer]] ***

For this question, either "right answer" or "correct answer" (not case sensitive) counts as correct.

*** ``` What is the right answer? [[right answer]] ***

For this question, either "right answer" or "correct answer" (not case sensitive) counts as correct.

*** This question accepts any of several items from a list of possible correct answers. It is not case sensitive (that's the little `i` at the end of the regex). With flexible answers like this, it's definitely a good idea to include a follow-up to help the learner put their answer in context. For example, if the question was "Name one or more colors" with acceptable answers including red, orange, yellow, green, blue, and purple, and they wrote "red, green, and the center of a black hole" that would be marked as correct because it contains at least one string from the acceptable list. Similarly, "hammered metal" would be marked as correct because it contains the string "red" ([you can prevent this if you want](https://www.w3schools.com/jsref/jsref_regexp_begin.asp)). On the other hand "teal, scarlet, indigo" would be marked wrong. ``` What are some items? [[this text will never show up if they type a right answer and click "Check", only if they click the checkmark button to reveal the answer]] [[?]] Hint: The answers are like "item1", "item2", etc. ***

Reiterate what the correct answer or answers should be, and try to anticipate likely wrong answers so you can explain why they're not correct.

*** ``` What are some items? [[this text will never show up if they type a right answer and click "Check", only if they click the checkmark button to reveal the answer]] [[?]] Hint: The answers are like "item1", "item2", etc. ***

Reiterate what the correct answer or answers should be, and try to anticipate likely wrong answers so you can explain why they're not correct.

*** Note that you can use any markdown formatting you want in quizzes, including bold, links, math, lists, embedded media, code blocks, etc. For accessibility for learners using screenreaders, if your question is more than a single markdown paragraph, you must enclose it in `

` tags. For example: ````

In the following code block, what should fill in the blank? ```r install.packages(____) ```

[(X)] "ggplot2" [( )] ggplot2 [( )] package = ggplot2 ***

Note that the `

` tags surround the full content of the question being asked (in this case, a sentence in markdown followed by the code block), and then there is one blank line, then the multiple choice options. For more details, see [notes about quizzes in the LiaScript documentation](https://liascript.github.io/course/?https://raw.githubusercontent.com/liaScript/do.cs/master/README.md#notes-about-questions)

*** ````

In the following code block, what should fill in the blank? ```r install.packages(____) ```

[(X)] "ggplot2" [( )] ggplot2 [( )] package = ggplot2 ***

Note that the `

*** Sometimes you may wish to include a quiz question that would be impossible to automatically grade (i.e. one with an open-ended response). To mark all possible answers as correct, use a brief script to match everything with regex: ``` Take a look at your samples. Which sample(s) looks the best in terms of per base sequence quality? Which sample(s) look the worst? [[describe sample quality]] ***

(Note that we can't automatically grade an open-ended question like this, so it will be marked "correct" no matter what you write.) All of the reads contain usable data, but the quality decreases toward the end of the reads.

*** ``` Take a look at your samples. Which sample(s) looks the best in terms of per base sequence quality? Which sample(s) look the worst? [[describe sample quality]] ***

*** There are many more options and examples of quiz questions in the LiaScript documentation. [Read more about quiz syntax here.](https://liascript.github.io/course/?https://raw.githubusercontent.com/liaScript/docs/master/README.md#quizzes) ## Ungraded quiz questions Useful formative assessment doesn't have to be an actual quiz question. You can ask questions with no graded answer as well. LiaScript calls these [surveys](https://liascript.github.io/course/?https://raw.githubusercontent.com/LiaScript/docs/master/README.md#111). Here's an ungraded question with a text box three lines long: ``` [[___ ___ ___]] ``` [[___ ___ ___]] Here's one that's just one line long: ``` [[___]] ``` [[___]] Hints and follow-up explanations don't work for survey questions, but you can use [html detail tags](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) to create a solution that won't be visible to the learner until they choose to open it. For example: ``` Think about your own research, and write out three ways you could make your work more reproducible this year. [[___ ___ ___]]

Click here to see our answer

The best approach for you depends on your research, of course! Here are some concrete, actionable changes to improve the reproducibility of your work: - Switch from point-and-click to scripted analysis - Share your data in a public repository - Share your code on GitHub - Write a pre-registration for your next study

``` Think about your own research, and write out three ways you could make your work more reproducible this year. [[___ ___ ___]]

Click here to see our answer

You can also set a code block to be closed until the learner chooses to open it by giving it a title beginning with `-` (e.g. `-Solution`). For example: ```` Modify the code from the final example, the [lowess curve trend line](#lowess-curve-trend-lines), to separate out respondents by smoking status (`is_smoker`) with a separate facet for each. ```python -Solution sns.lmplot(data = covid_data, x="val_age", y="val_height_cm", scatter_kws={"alpha": .1}, lowess=True, col = "is_smoker") # Note that row = "is_smoker" would also work. # If you used col, try switching to row now to see how the plot changes! ``` ```` Modify the code from the final example, the [lowess curve trend line](#lowess-curve-trend-lines), to separate out respondents by smoking status (`is_smoker`) with a separate facet for each. ```python -Solution sns.lmplot(data = covid_data, x="val_age", y="val_height_cm", scatter_kws={"alpha": .1}, lowess=True, col = "is_smoker") # Note that row = "is_smoker" would also work. # If you used col, try switching to row now to see how the plot changes! ``` You can even use this to provide multiple possible correct options. Each solution will be closed until the learner clicks to open it. For example: ```` Write code to draw a linear trend line showing the relationship between Age and Glucose, but create a plot with just the line, no scatterplot underneath. Try it each of the three ways, using geom\_smooth, geom\_abline, and geom\_line. ```r -Solution using geom_smooth ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_smooth(method = "lm") + theme_bw() ``` ```r -Solution using geom_abline # note that this doesn't actually plot a line, since there are no observations to set the x and y scales # you'll see a blank plot ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_abline(intercept = model$coefficients[1], slope = model$coefficients[2]) + theme_bw() # you can set the x and y scales yourself manually by adding a layer for each ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_abline(intercept = model$coefficients[1], slope = model$coefficients[2]) + scale_y_continuous(limits = c(min(breast_cancer_data$Glucose), max(breast_cancer_data$Glucose))) + scale_x_continuous(limits = c(min(breast_cancer_data$Age), max(breast_cancer_data$Age))) + theme_bw() ``` ```r -Solution using geom_line ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_line(mapping = aes(y=model$fitted.values)) + theme_bw() ``` ```r -Another solution, using alpha # you can also make any element of a plot invisible by setting its alpha to 0 # in this case, we can make the dots of the scatterplot disappear from any of the plots we made above # for example: ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_point(alpha = 0) + geom_line(mapping = aes(y=model$fitted.values)) + theme_bw() # this has the advantage of keeping the scales for the plot consistent # and it means you don't have to set the scales manually when using geom_abline ``` ```` Write code to draw a linear trend line showing the relationship between Age and Glucose, but create a plot with just the line, no scatterplot underneath. Try it each of the three ways, using geom\_smooth, geom\_abline, and geom\_line. ```r -Solution using geom_smooth ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_smooth(method = "lm") + theme_bw() ``` ```r -Solution using geom_abline # note that this doesn't actually plot a line, since there are no observations to set the x and y scales # you'll see a blank plot ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_abline(intercept = model$coefficients[1], slope = model$coefficients[2]) + theme_bw() # you can set the x and y scales yourself manually by adding a layer for each ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_abline(intercept = model$coefficients[1], slope = model$coefficients[2]) + scale_y_continuous(limits = c(min(breast_cancer_data$Glucose), max(breast_cancer_data$Glucose))) + scale_x_continuous(limits = c(min(breast_cancer_data$Age), max(breast_cancer_data$Age))) + theme_bw() ``` ```r -Solution using geom_line ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_line(mapping = aes(y=model$fitted.values)) + theme_bw() ``` ```r -Another solution, using alpha # you can also make any element of a plot invisible by setting its alpha to 0 # in this case, we can make the dots of the scatterplot disappear from any of the plots we made above # for example: ggplot(breast_cancer_data, mapping = aes(y=Glucose, x=Age)) + geom_point(alpha = 0) + geom_line(mapping = aes(y=model$fitted.values)) + theme_bw() # this has the advantage of keeping the scales for the plot consistent # and it means you don't have to set the scales manually when using geom_abline ``` ## Updating this document **This is version @version of this document.** When you update docs.md, be sure to [increment its version](https://github.com/arcus/education_modules/blob/main/versioning_guidelines.md) in the front matter. For any **major** updates to docs.md, you should also provide an issue template checklist for bringing modules up to date with the new version of docs. For example, see the [checklist for docs v1](https://github.com/arcus/education_modules/blob/main/.github/ISSUE_TEMPLATE/docs_update_checklist_v1.md). This will create a new option in our list of available issues (see [create a new issue](https://github.com/arcus/education_modules/issues/new/choose)), which we can then use to bring all of the current modules up to date with the new guidelines. If a module misses a round of updates (e.g. it never gets updated for docs 1.0.0 but there's already a docs 2.0.0 out), then it will need to go through all of the checklists to bring it up to date.

Important note
If you're planning major updates to docs.md, it's a good idea to start your checklist issue template right at the beginning of your editing process and add to it gradually as you make changes in docs.md so you don't forget anything.