Horje
how to select specific column with Dimensionality Reduction pyspark Code Example
how to select specific column with Dimensionality Reduction pyspark
import org.apache.spark.mllib.linalg.Matrix
import org.apache.spark.mllib.linalg.SingularValueDecomposition
import org.apache.spark.mllib.linalg.Vector
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.linalg.distributed.RowMatrix

val data = Array(
  Vectors.sparse(5, Seq((1, 1.0), (3, 7.0))),
  Vectors.dense(2.0, 0.0, 3.0, 4.0, 5.0),
  Vectors.dense(4.0, 0.0, 0.0, 6.0, 7.0))

val rows = sc.parallelize(data)

val mat: RowMatrix = new RowMatrix(rows)

// Compute the top 5 singular values and corresponding singular vectors.
val svd: SingularValueDecomposition[RowMatrix, Matrix] = mat.computeSVD(5, computeU = true)
val U: RowMatrix = svd.U  // The U factor is a RowMatrix.
val s: Vector = svd.s     // The singular values are stored in a local dense vector.
val V: Matrix = svd.V     // The V factor is a local dense matrix.
how to select specific column with Dimensionality Reduction pyspark
from pyspark.mllib.linalg import Vectors
from pyspark.mllib.linalg.distributed import RowMatrix

rows = sc.parallelize([
    Vectors.sparse(5, {1: 1.0, 3: 7.0}),
    Vectors.dense(2.0, 0.0, 3.0, 4.0, 5.0),
    Vectors.dense(4.0, 0.0, 0.0, 6.0, 7.0)
])

mat = RowMatrix(rows)

# Compute the top 5 singular values and corresponding singular vectors.
svd = mat.computeSVD(5, computeU=True)
U = svd.U       # The U factor is a RowMatrix.
s = svd.s       # The singular values are stored in a local dense vector.
V = svd.V       # The V factor is a local dense matrix.




Python

Related
how to get last item of a list in Python Code Example how to get last item of a list in Python Code Example
generate a list of random numbers between 1 and 100 python Code Example generate a list of random numbers between 1 and 100 python Code Example
how many numbers greater than 100 using pytho Code Example how many numbers greater than 100 using pytho Code Example
Unpacking list using an asterisk Code Example Unpacking list using an asterisk Code Example
dorp ligne in df where values equal zeros Code Example dorp ligne in df where values equal zeros Code Example

Type:
Code Example
Category:
Coding
Sub Category:
Code Example
Uploaded by:
Admin
Views:
11