autoweka

## Class Experiment

- java.lang.Object
- - autoweka.XmlSerializable
  - - autoweka.Experiment

-     public class Experiment
      extends XmlSerializable

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `Experiment.TrajectoryPointExtra` Class capturing any of the extra dataset partitions that should be run on all the trajectory points for analysis

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `java.util.List<java.lang.String>` | `allowedClassifiers` Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can
    `boolean` | `attributeSelection` Boolean indicating if Attribute/Feature selection should happen
    `float` | `attributeSelectionTimeout` How many seconds should be spent performing attribute selection?
    `java.util.List<java.lang.String>` | `callString` The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor
    `java.lang.String` | `datasetString` The string that indicates what dataset is going to be used
    `java.util.List<java.lang.String>` | `envVariables` A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor
    `java.lang.String` | `extraPropsString` Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers
    `java.lang.String` | `instanceGenerator` The class name of the instance generator
    `java.lang.String` | `instanceGeneratorArgs` The property string that contains the construction arguments for the instance generator
    `java.lang.String` | `memory` The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have
    `java.lang.String` | `name` The name of the experiment, designed to be human readable and contain useful ID information
    `java.lang.String` | `resultMetric` The name of the result metric to use - see ClassiferResult.Metric
    `float` | `trainTimeout` How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?.
    `java.lang.String` | `trajectoryParserClassName` The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor
    `java.util.List<Experiment.TrajectoryPointExtra>` | `trajectoryPointExtras` For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory.
    `float` | `tunerTimeout` How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)
    `java.lang.String` | `type` A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Experiment()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static Experiment` | `createFromFolder(java.io.File folder)`
    `static Experiment` | `fromXML(java.io.InputStream xml)` Static method to load up an experiment from XML.
    `static Experiment` | `fromXML(java.lang.String filename)` Static method to load up an experiment from XML.
    `static void` | `main(java.lang.String[] args)` Main method to actually run an experiment with a given seed.
    `void` | `validate()` Throws a runtime exception if the experiment contains some sort of crazy values.

    - ### Methods inherited from class autoweka.XmlSerializable

      `fromXML, fromXML, populateObjectFromCMDParams, populateObjectFromCMDParams, toXML, toXML`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### name

          public java.lang.String name

      The name of the experiment, designed to be human readable and contain useful ID information

    - #### type

          public java.lang.String type

      A helper variable that indicates the type of the experiment - this is set by the ExperimentConstructor

    - #### resultMetric

          public java.lang.String resultMetric

      The name of the result metric to use - see ClassiferResult.Metric

    - #### instanceGenerator

          public java.lang.String instanceGenerator

      The class name of the instance generator

    - #### instanceGeneratorArgs

          public java.lang.String instanceGeneratorArgs

      The property string that contains the construction arguments for the instance generator

    - #### datasetString

          public java.lang.String datasetString

      The string that indicates what dataset is going to be used

    - #### attributeSelection

          public boolean attributeSelection

      Boolean indicating if Attribute/Feature selection should happen

    - #### attributeSelectionTimeout

          public float attributeSelectionTimeout

      How many seconds should be spent performing attribute selection?

    - #### trajectoryParserClassName

          public java.lang.String trajectoryParserClassName

      The name of the class that's going to be used to parse the trajectory - this is set by the ExperimentConstructor

    - #### callString

          public java.util.List<java.lang.String> callString

      The list of command line arguments that are used to invoke the SMBO method - this is set by the ExperimentConstructor

    - #### envVariables

          public java.util.List<java.lang.String> envVariables

      A list of var=values that are set in environment variables before the SMBO method is executed - these are set by the ExperimentConstructor

    - #### tunerTimeout

          public float tunerTimeout

      How many seconds should be spent for this experiment overall? (IE the SMBO method's budget)

    - #### trainTimeout

          public float trainTimeout

      How many seconds should be spent training a set of hyperparameters on a specific partition of the training and test data?. Note that this is more of a guideline, Auto-WEKA has quite a bit of slack built into to accomodate classifiers that can return partial results, or ones that decide to offload training onto their evaluation phase.

    - #### memory

          public java.lang.String memory

      The string passed to the Xmx argument of a sub process limiting the RAM that WEKA will have

    - #### extraPropsString

          public java.lang.String extraPropsString

      Any extra properties that are associated with the experiment, these are generally for the constructor or your wrappers

    - #### allowedClassifiers

          public java.util.List<java.lang.String> allowedClassifiers

      Forces Auto-WEKA to only use the list of classifiers here - if it is empty, then Auto-WEKA will try to use everything that it can

    - #### trajectoryPointExtras

          public java.util.List<Experiment.TrajectoryPointExtra> trajectoryPointExtras

      For analysis experiements, these extra classifier evaluations will be done for every point along the trajectory. For example, if we want to look at the Testing performance all the way along, we would add a new TrajectoryPointExtra that contains an instance string of 'default', since that causes Auto-WEKA to preserve the provided training/test split

  - ### Constructor Detail

    - #### Experiment

          public Experiment()

  - ### Method Detail

    - #### validate

          public void validate()

      Throws a runtime exception if the experiment contains some sort of crazy values.

    - #### fromXML

          public static Experiment fromXML(java.lang.String filename)

      Static method to load up an experiment from XML.
      Parameters:
        `filename` - The name of the file.
      Returns:
        The experiment.

    - #### fromXML

          public static Experiment fromXML(java.io.InputStream xml)

      Static method to load up an experiment from XML.
      Parameters:
        `xml` - The input stream.
      Returns:
        The experiment.

    - #### main

          public static void main(java.lang.String[] args)

      Main method to actually run an experiment with a given seed. This requires two command line arguments, the path to the experiment folder (but not the actual .experiment itself) and the seed on which you want to run this experiment. The seed is passed to the SMBO method to allow for easy paralellization of each SMBO method.
      Parameters:
        `args` - Any arguments.

    - #### createFromFolder

          public static Experiment createFromFolder(java.io.File folder)