spark

Type Members

1. trait SparkExecutionContext extends ExecutionContext

   :: AlpineSdkApi ::

2. abstract class SparkIOTypedPluginJob[I, O] extends AnyRef

   :: AlpineSdkApi ::

3. case class SparkJobConfiguration(numExecutors: Int, executorMemoryMB: Int, driverMemoryMB: Int, numExecutorCores: Int, additionalParameters: Map[String, String] = ...) extends Product with Serializable

   Spark job configuration.

4. abstract class SparkRuntime[I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   Defines the behavior of your plugin after the end user 'runs' it from the GUI.

5. abstract class SparkRuntimeWithIOTypedJob[J <: SparkIOTypedPluginJob[I, O], I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   A descendant of SparkRuntime which handles the most straightforward runtime behavior, submitting a Spark job with the input of your plugin and returning the output type.

6. trait SubmittedSparkJob[O] extends AnyRef

   :: AlpineSdkApi ::