MLlibUtils

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. def anyToDouble(a: Any): Double

   

   Converts input of type Any to Double.

   Converts input of type Any to Double. Does this by casting to java.lang.Number, and then taking the double value.

   Will return Double.NaN in the case of bad input.

   a

   input to be converted to Double.

   returns

   Double representation of the number, or Double.NaN if impossible.

5. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

6. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

7. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

8. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

9. def finalize(): Unit

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( classOf[java.lang.Throwable] )

10. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

11. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

12. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

13. def mapSeqToCorrectType[T <: Seq[Any]](rowRDD: RDD[T], schema: StructType, nullable: Boolean = true): RDD[Row]

    

    An ugly function that handles the case where data that was a long, has been converted to a double and then can't be converted back i.e.

    An ugly function that handles the case where data that was a long, has been converted to a double and then can't be converted back i.e. 10 becomes 10.0 and then creating a dataFrame with an integer or long column fails. A non numeric input in a double or integer column will be converted to null, since round of Double.NAN is 0. Set the value of the nullable parameter to true in order to keep the behavior where null values in Integer and Long columns are converted to zeros.

    rowRDD

    and rdd of some sequence type that will be the body of the DataFrame

    schema

    the schema that we want tthe new RDD to conform to

    returns

    rowRDD with each value parsed so it won't cause failures when sqlContext.createDataFrame(rowRDD, schema) is called

14. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

15. final def notify(): Unit

    

    Definition Classes

    AnyRef

16. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

17. def rowToDoubleVector(row: Row): Vector

    

18. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

19. def toLabeledPoint(dependentColumnIndex: Int, independentColumnIndices: Seq[Int]): (Row) ⇒ LabeledPoint

    
    dependentColumnIndex

    Index of the dependent column in the column set.

    independentColumnIndices

    Indices of the independent columns.

    returns

    Function mapping a Row of the correct schema to a labelled point.

20. def toString(): String

    

    Definition Classes

    AnyRef → Any

21. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

22. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

23. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )