{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Creating a sky flat\n",
"\n",
"One way of producing flat field images is to use the science images, combined in\n",
"a way that eliminates astronomical sources. This provides an exact match to the\n",
"spectrum of the night sky, since the night sky is the source of light. However,\n",
"the night sky is dark, so the counts in individual images is low. Many images\n",
"must be combined to generate a flat with low noise."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"import numpy as np\n",
"\n",
"from astropy.nddata import CCDData\n",
"from astropy import units as u\n",
"from astropy.stats import mad_std\n",
"\n",
"from photutils import detect_threshold, detect_sources, source_properties\n",
"\n",
"import ccdproc as ccdp"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## When is it impossible to produce a sky flat?\n",
"\n",
"There are a few circumstances in which producing a sky flat is difficult or\n",
"impossible:\n",
"\n",
"+ The telescope tracks very well so stars and other sources are always in\n",
"roughly the same pixels in all of the images. In this case, there is no way to\n",
"produce a good flat. If several fields of view are observed this should not be\n",
"an issue.\n",
"+ There is an extended source that covers an appreciable fraction of the field\n",
"of view. In this case there is likely to be overlap of the extended object\n",
"between images, so it cannot be removed from the flat.\n",
"+ The sky is really dark. In very dark sites the sky background might be low\n",
"enough that the sky flat is too noisy to be useful."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Producing a sky flat\n",
"\n",
"Producing a sky flat is much like producing any other flat. The images must have\n",
"bias and dark current subtracted (and overscan if it is being used) then\n",
"combined, rescaling each image to take into account different levels of\n",
"background illumination.\n",
"\n",
"It is important to scale the *median* of each image to the same value instead of\n",
"scaling the *mean* because the presence of bright sources will affect the mean\n",
"much more than the median.\n",
"\n",
"One down side of producing sky flats is the need to process the science images\n",
"twice. The first time all of the usual calibration steps except flat fielding\n",
"are done, then the flats are produced, then each science image is flat\n",
"corrected."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Partially calibrate science images\n",
"\n",
"The partially reduced images are saved in a different folder than the completely\n",
"reduced science images that were processed earlier.\n",
"\n",
"The images for this example were taken the same night as the other images in\n",
"\"Example 2\" in earlier notebooks.\n",
"\n",
"First, we set up some of the locations we will need."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ex2_calibrated = Path('example2-reduced')\n",
"\n",
"sky_flat_bad_raw = Path('sky_flat_good_raw')\n",
"\n",
"sky_flat_bad_working = Path('sky_flat_good_working')\n",
"sky_flat_bad_working.mkdir(exist_ok=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, load the combined bias and combined dark for this night. Recall that the\n",
"combined dark for this night was bias-subtracted because it needed to be scaled\n",
"for the flat images (see [this notebook](03.05-Calibrate-dark-images.ipynb#Example-2:-Overscan-not-subtracted,-bias-is-removed) for more detail).\n",
"\n",
"All of the science exposures this night had the same exposure time, 90 sec."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"combined_bias = CCDData.read(ex2_calibrated / 'combined_bias.fit')\n",
"combined_dark = CCDData.read(ex2_calibrated / 'combined_dark_90.000.fit')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The telescope tracking changed during this night. Tracking was excellent for\n",
"observations of Kelt 16b, making the images terrible for sky flats, but\n",
"excellent for illustrating the failure of sky flats under some circumstances."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"ifc_raw = ccdp.ImageFileCollection(sky_flat_bad_raw)\n",
"\n",
"for ccd, name in ifc_raw.ccds(imagetyp='light', object='wasp 10 b', filter=\"r\", return_fname=True):\n",
" reduced = ccdp.trim_image(ccd[:, :4096])\n",
" reduced = ccdp.subtract_bias(reduced, combined_bias)\n",
" reduced = ccdp.subtract_dark(reduced, combined_dark, exposure_time='exposure', exposure_unit=u.second)\n",
" thresh = detect_threshold(an_im, 2)\n",
" segm = detect_sources(an_im, thresh, 30)\n",
" reduced.data[segm.data > 0] = np.nan\n",
" reduced.data = reduced.data.astype('float32')\n",
" reduced.write(sky_flat_bad_working / name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Combine the partially calibrated images\n",
"\n",
"The combination settings here are important. Either combine by averaging and\n",
"sigma clip or combine by median. Either should ensure that stars do not show up\n",
"in your final flat as long as there is enough offset between the images. Images\n",
"need to be scaled so that the median is the same for each image. Typically, a\n",
"value of one is chosen as the common value."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ifc_working = ccdp.ImageFileCollection(sky_flat_bad_working)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"to_combine = [ccd for ccd in ifc_working.ccds()]\n",
"\n",
"def inv_median(array):\n",
" return 1 / np.nanmedian(array)\n",
"\n",
"sky_flat = ccdp.combine(to_combine, scale=inv_median, \n",
" sigma_clip=True, sigma_clip_low_thresh=3, sigma_clip_high_thresh=3,\n",
" sigma_clip_func=np.nanmedian, sigma_clip_dev_func=mad_std, \n",
" mem_limit=2e9\n",
" )\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from convenience_functions import show_image"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"an_im = CCDData.read(sky_flat_bad_working / 'wasp-10-b-S001-R001-C050-r.fit')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"show_image(an_im, cmap='gray')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from photutils import detect_threshold, detect_sources, source_properties"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"foo = detect_threshold(an_im, 2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arf = detect_sources(an_im, foo, 30)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"show_image(arf.data > 0, cmap='gray', is_mask=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"(arf.data > 0).sum()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"moo = source_properties(an_im.data, arf)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"moo.to_table()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"show_image(sky_flat, cmap='gray')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sky_flat.write('supposed_to_be_good_but_has_streaks.fits')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}