spark

Type Members

1. final class OperatorFailedException extends Exception

   To be thrown if the operator encounters some exception at runtime (like a bad configuration), and the developer does not want the framework (in particular, Spark) to retry the operation.

2. trait SparkExecutionContext extends ExecutionContext

   :: AlpineSdkApi ::

3. abstract class SparkIOTypedPluginJob[I, O] extends AnyRef

   :: AlpineSdkApi ::

4. case class SparkJobConfiguration(numExecutors: Int, executorMemoryMB: Int, driverMemoryMB: Int, numExecutorCores: Int, additionalParameters: Map[String, String] = ...) extends Product with Serializable

   Spark job configuration.

5. abstract class SparkRuntime[I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   Defines the behavior of your plugin after the end user 'runs' it from the GUI.

6. abstract class SparkRuntimeWithIOTypedJob[J <: SparkIOTypedPluginJob[I, O], I <: IOBase, O <: IOBase] extends OperatorRuntime[SparkExecutionContext, I, O]

   A descendant of SparkRuntime which handles the most straightforward runtime behavior, submitting a Spark job with the input of your plugin and returning the output type.

7. trait SubmittedSparkJob[O] extends AnyRef

   :: AlpineSdkApi ::