# Earth Lens *Please note that Microsoft will not officially support this repository starting Feb 28, 2019. However, contributions from the community are still welcome!* [Earth Lens](https://github.com/Microsoft/Earth-Lens) leverages Microsoft’s AI tools to detect objects in aerial imagery and translate raw image data into digestible insights for researchers. With this tool, researchers will be able to automatically extract information from a large dataset and have the analysis presented in a useful and interactive view. As a result, this tool can be useful in various industrial or agricultural applications to further Microsoft’s AI For Earth commitment to sustainability. The application utilizes Apple's [CoreML framework](https://developer.apple.com/documentation/coreml) to run the machine learning model locally on the iPad, showcasing the power and capability of the Intelligent Edge. Users are able to upload images, process the locally without the internet, and then scroll through and analyze the data. Earth Lens helps users automatically aggregate, count, classify and label the objects in satellite images. Colored bounding boxes are drawn around detected objects, which users can toggle on and off to control the information displayed. A time series analysis view allows users to look at images over time alongside a data visualization chart to identify trends and patterns. # Device Compatibility This application runs on iPad devices with iOS 9.0 or above. # Features * Categorization / Filtering Sidebar * Timeline Series Analysis with Data Visualizer * Color Customization * Observation Relabeling * PDF Report Export * Customized Settings # Project Structure ## Xamarin This folder contains the application Visual Studio solution, which consists of five separate projects: ### EarthLens * Code that is platform-independent and common to multiple platforms * Class definitions that provide API for the application to represent CoreML model outputs * Observations detected by the CoreML model * Categories and mega-categories that classify detected observations * Chip and image entry for intermediate representations * Internal gallery section * Database * Database access objects (DAO) of images and observations * CoreML model * Database service to manage the database (e.g. remove / insert entries) * Environment service to manage environment variables for App Center * Graphic service for PDF report generation and export * Image analysis service that utilizes parallel computing to analyze images * Image encoding service to convert images to their base64 representations * JSON service to serialize and deserialize JSON format * Machine learning service for mathmetical functions related to ML * Post processing service to apply filtering and non-maximum suppression algorithm to results detected by the CoreML model ### EarthLens.iOS * Core iOS application code and UI * Storyboards files, XIB files, etc. with view controllers * Icon assets * CoreML-based implementation of for the machine learning model integration * PDF generation service for PDF report export * Data visualization service to generate bar charts data visualizer ### EarthLens.Tests * Unit testing project for the `EarthLens` project ### EarthLens.Tests.iOS * Unit testing project for the `EarthLens.iOS` project ### EarthLens.TestUtils * Common unit testing utility code that is common to both `EarthLens.Tests` and `EarthLens.Tests.iOS` # Getting Started with Development ## Setup on Windows ### Windows 7 or above 1. Update your Windows if you are on a version older than Windows 7, as specified in the Xamarin [System requirements](https://docs.microsoft.com/en-us/xamarin/cross-platform/get-started/requirements). ### Visual Studio 2017 (Community, Professional, or Enterprise) and Xamarin 4.5.0 or above 1. Follow the [Windows installation instructions](https://docs.microsoft.com/en-us/xamarin/cross-platform/get-started/installation/windows#Installation) to install Visual Studio 2017 (Community, Professional, or Enterprise) and the Xamarin component on Windows. ### Mac Agent 1. Follow the official Xamarin documentation to [Pair to Mac for Xamarin.iOS Development](https://docs.microsoft.com/en-us/xamarin/ios/get-started/installation/windows/connecting-to-mac/). ## Setup on macOS ### macOS High Sierra 10.12 or above 1. Update your OS version if you are on a version older than 10.12, as specified in the Xamarin [System requirements](https://docs.microsoft.com/en-us/xamarin/cross-platform/get-started/requirements). ### Xcode 9 or above 1. Download and install Xcode from App Store on macOS. 2. Launch Xcode. 3. Read and accept the License Agreement. 4. Let Xcode install additional components (this includes command-line tools that Visual Studio needs). ### Visual Studio for Mac 1. Follow the official Microsoft documentation to [Setup and Install Visual Studio for Mac](https://docs.microsoft.com/en-us/visualstudio/mac/installation) * Make sure at least the following options are selected when choosing what to install * iOS * .NET Core ## Getting the Source Code 1. Clone the repository to your development machine: ```Bash git clone https://github.com/Microsoft/Earth-Lens.git ``` 2. Open `Xamarin/EarthLens.sln` in Visual Studio (for Windows) or Visual Studio for Mac (for macOS). ## Building/Running the App ### Running on a simulator * Note for Windows users: * You can use a [Remoted iOS Simulator](https://docs.microsoft.com/en-us/xamarin/tools/ios-simulator) that is remotely connected to the Mac Agent and will launch on the Windows machine. * If you choose not to use a remoted iOS simulator, the simulator will launch on the Mac Agent. 1. In Visual Studio, select the following configuration: | | | | ------------- | ----------------- | | Project | `EarthLens.iOS` | | Configuration | `Debug` | | Platform | `iPhoneSimulator` | | Device | `