spark

Type Members

1. case class AutoTunerOptions(options: Map[String, String]) extends Product with Serializable

2. final class OperatorFailedException extends Exception

   To be thrown if the operator encounters some exception at runtime (like a bad configuration), and the developer does not want the framework (in particular, Spark) to retry the operation.

3. trait SparkExecutionContext extends ExecutionContext

   :: AlpineSdkApi ::

4. abstract class SparkIOTypedPluginJob[I, O] extends AnyRef

   :: AlpineSdkApi ::

5. case class SparkJobConfiguration(userDefinedParameters: Map[String, String], autoTuneMissingValues: Boolean, autoTunerOptions: AutoTunerOptions) extends Product with Serializable

   Spark job configuration.

6. abstract class SparkRuntime[I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   Defines the behavior of your plugin after the end user 'runs' it from the GUI.

7. abstract class SparkRuntimeWithIOTypedJob[J <: SparkIOTypedPluginJob[I, O], I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   A descendant of SparkRuntime which handles the most straightforward runtime behavior, submitting a Spark job with the input of your plugin and returning the output type.

8. trait SubmittedSparkJob[O] extends AnyRef

   :: AlpineSdkApi ::