SparkParameterUtils

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. def addStandardSparkOptions(operatorDialog: OperatorDialog, defaultNumExecutors: Int, defaultExecutorMemoryMB: Int, defaultDriverMemoryMB: Int, defaultNumExecutorCores: Int): Unit

7. def addStandardSparkOptionsWithStorageLevel(operatorDialog: OperatorDialog, defaultNumExecutors: Int, defaultExecutorMemoryMB: Int, defaultDriverMemoryMB: Int, defaultNumExecutorCores: Int, defaultStorageLevel: String, additionalSparkParameters: List[SparkParameter] = List.empty[SparkParameter]): Unit

   A more advanced method for adding SparkP Parameters.

   A more advanced method for adding SparkP Parameters. Will also add a "StorageLevel" Parameter which will indicate what level of persistence to use within a Spark job. NOTE: The storage level parameter cannot be set automatically during runtime. To have any effect the custom operator developer must implement RDD persistence with this value (retrievable with 'getStorageLevel' method) in the Spark Job class of their operator.

   defaultStorageLevel

   - default storage level e.g. NONE or "MEMORY_AND_DISK.

   additionalSparkParameters

   - a list of a additional Spark Parameters.

8. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

9. def clone(): AnyRef

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

10. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

11. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

12. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

13. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

14. def getStorageLevel(operatorParameters: OperatorParameters): Option[String]

15. def hashCode(): Int

    Definition Classes

    AnyRef → Any

16. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

17. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

18. final def notify(): Unit

    Definition Classes

    AnyRef

19. final def notifyAll(): Unit

    Definition Classes

    AnyRef

20. val sparkDriverMBElementId: String

21. val sparkExecutorMBElementId: String

22. val sparkNumExecutorCoresElementId: String

23. val sparkNumExecutorsElementId: String

24. val storageLevelParamId: String

25. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

26. def toString(): String

    Definition Classes

    AnyRef → Any

27. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

28. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

29. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )