MLlibUtils

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. def anyToDouble(a: Any): Double

   Converts input of type Any to Double.

   Converts input of type Any to Double. Does this by casting to java.lang.Number, and then taking the double value.

   Will return Double.NaN in the case of bad input.

   a

   input to be converted to Double.

   returns

   Double representation of the number, or Double.NaN if impossible.

7. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

8. def clone(): AnyRef

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

9. final def eq(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

10. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

11. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

12. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

13. def hashCode(): Int

    Definition Classes

    AnyRef → Any

14. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

15. def mapSeqToCorrectType[T <: Seq[Any]](rowRDD: RDD[T], schema: StructType, nullable: Boolean = true): RDD[Row]

    An ugly function that handles the case where data that was a long, has been converted to a double and then can't be converted back i.

    An ugly function that handles the case where data that was a long, has been converted to a double and then can't be converted back i.e. 10 becomes 10.0 and then creating a dataFrame with an integer or long column fails. A non numeric input in a double or integer column will be converted to null, since round of Double.NAN is 0. Set the value of the nullable parameter to true in order to keep the behavior where null values in Integer and Long columns are converted to zeros.

    rowRDD

    and rdd of some sequence type that will be the body of the DataFrame

    schema

    the schema that we want tthe new RDD to conform to

    returns

    rowRDD with each value parsed so it won't cause failures when sqlContext.createDataFrame(rowRDD, schema) is called

16. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

17. final def notify(): Unit

    Definition Classes

    AnyRef

18. final def notifyAll(): Unit

    Definition Classes

    AnyRef

19. def rowToDoubleVector(row: Row): Vector

20. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

21. def toLabeledPoint(dependentColumnIndex: Int, independentColumnIndices: Seq[Int]): (Row) ⇒ LabeledPoint

    dependentColumnIndex

    Index of the dependent column in the column set.

    independentColumnIndices

    Indices of the independent columns.

    returns

    Function mapping a Row of the correct schema to a labelled point.

22. def toString(): String

    Definition Classes

    AnyRef → Any

23. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

24. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

25. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )