{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "

GESIS Summer School in Survey Methodology 2018:
Meta-Analysis in Social Research and Survey Methodology

" ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preliminaries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Please do not touch anything in this section, otherwise this notebook might not work properly. You have been warned! Also, if you have no clue what you are staring at, please consult our [Preface chapter](1-1_preface.ipynb)." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "source(\"run_me_first.R\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this exercise, we want you to estimate the fixed-effect and the random-effects model for the `dat.bcg` data (using risk ratios).\n", "\n", "Next, we would like you to manually estimate the “common” effect, i.e. the summary effect according to the fixed-effect model, its variance/standard error and the 95% CI. You will need the formulas and the slide “Summary” in the subsection ‘Distinction between fixed- and random-effects’ might be useful to get you started. Tip: The function `sum(x)` calculates the sum of a vector `x`. \n", "\n", "Note: you will:\n", "1. compute the weights under fixed effect models (tip: `wi <- 1/dat.bcg$vi`)\n", "2. calculate the weighted mean (tip: `ma.fem <- sum(wi * dat.bcg$yi) / sum(wi)`)\n", "3. calculate the variance (tip: `var.ma.fem <- 1/sum(wi)`)\n", "4. calculate the standard error (tip: `se.ma.fem <- sqrt(var.ma.fem)`)\n", "5. finally, compute the 95% CI (tip: for the lower bound `ma.fem.ll <- ma.fem - 1.96 * se.ma.fem`)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course, as you may already know, you do not have to estimate summary effects by hand. `metafor`'s `rma()` function can do that for you.\n", "\n", "For this exercise, you can apply the `rma()` function to replicate the results from exercise 1, i.e. estimate a fixed-effect summary. If you forgot how to apply the `?rma` function, check the metafor introduction slides and exercises. (Remember, you only need for now to know three arguments in the ?rma function: the effect-size argument `yi` , the variance argument `vi`, and the argument `method`. With the latter, you specify which effect-size model you want to apply.) Here, we need the fixed-effect model and therefore the argument is `method = \"FE\"`. \n", "\n", "Note: See the slides from the morning, the command that you need were used in the example, but be sure to\n", "use the `method = \"FE\"` for this exercise." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we will estimate the random-effects model. This morning, we have discussed that there exist several\n", "between-study variance estimates and we introduced the method-of-moments estimator (also known as DerSimonian and Laird estimator). In `metafor`’s `rma()` function, this can be accomplished with the `method = \"DL\"` argument.\n", "\n", "So, estimate the random-effects model and discuss the following questions:\n", "1. How large is the difference between the FE and the RE estimate?\n", "2. Compare the confidence intervals of both estimates. Which one is (should be) larger and why?\n", "3. Which model would you choose?" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", "Random-Effects Model (k = 13; tau^2 estimator: EB)\n", "\n", "tau^2 (estimated amount of total heterogeneity): 0.2682 (SE = 0.1801)\n", "tau (square root of estimated tau^2 value): 0.5178\n", "I^2 (total heterogeneity / total variability): 87.49%\n", "H^2 (total variability / sampling variability): 7.99\n", "\n", "Test for Heterogeneity: \n", "Q(df = 12) = 85.8625, p-val < .0001\n", "\n", "Model Results:\n", "\n", "estimate se zval pval ci.lb ci.ub \n", " -0.5429 0.1842 -2.9474 0.0032 -0.9040 -0.1819 **\n", "\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 \n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Solution according to http://www.metafor-project.org/doku.php/analyses:berkey1995.\n", "## IMPORTANT: Different metehod (\"EB\") applied! \n", "library(metafor)\n", "dat.bcg <- escalc(measure = \"RR\", ai = tpos, bi = tneg, ci = cpos, di = cneg, data = dat.bcg)\n", "dat.bcg$vi <- with(dat.bcg, sum(tneg/tpos)/(13*(tneg+tpos)) + \n", " sum(cneg/cpos)/(13*(cneg+cpos)))\n", "ma.rem <- rma(yi = yi, vi = vi, method = \"EB\", data = dat.bcg)\n", "ma.rem" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For those of you who are already bored, we suggest that you try estimating the RE model by hand. \n", "\n", "Note: to estimate between-studies variability you will need to compute $Q_T$ and use this information to estimate\n", "the $\\tau^2_{DL}$. Formulas are provided in our slides." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "## Please insert your solution here. Of course, feel free to add new code cells." ] } ], "metadata": { "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "3.5.1" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 2 }