com.cerner.bunsen.spark

Class Bundles

java.lang.Object

com.cerner.bunsen.spark.Bundles

public class Bundles
extends Object

Utilities to work with FHIR bundles.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Bundles.BundleContainer A wrapper class for bundles that supports the use of Bundles from various FHIR versions in Spark RDDs.

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	extractEntry(org.apache.spark.sql.SparkSession spark, org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles, Class resourceClass) Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	extractEntry(org.apache.spark.sql.SparkSession spark, org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles, Class resourceClass, List<Class> containedClasses) Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type, including any declared resources contained to the parent resource.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	extractEntry(org.apache.spark.sql.SparkSession spark, org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles, String resourceTypeUrl) Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type.
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	extractEntry(org.apache.spark.sql.SparkSession spark, org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles, String resourceTypeUrl, List<String> containedClassesUrls) Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type, including any declared resources contained to the parent resource.
static Bundles	forR4()
static Bundles	forStu3()
org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer>	fromJson(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> jsonBundles, String column) Returns an RDD of bundles loaded from the given dataset that has JSON-encoded bundles in the given column.
org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer>	fromJson(org.apache.spark.sql.Dataset<String> jsonBundles) Returns an RDD of bundles loaded from the given dataset of JSON-encoded bundles.
org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer>	fromXml(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> xmlBundles, String column) Returns an RDD of bundles loaded from the given dataset that has XML-encoded bundles in the given column.
org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer>	fromXml(org.apache.spark.sql.Dataset<String> xmlBundles) Returns an RDD of bundles loaded from the given dataset of XML-encoded bundles.
org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer>	loadFromDirectory(org.apache.spark.sql.SparkSession spark, String path, int minPartitions) Returns an RDD of bundles loaded from the given path.
void	saveAsDatabase(org.apache.spark.sql.SparkSession spark, org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles, String database, String... resourceNames) Saves an RDD of bundles as a database, where each table has the resource name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

forStu3

public static Bundles forStu3()

forR4

public static Bundles forR4()

loadFromDirectory

public org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> loadFromDirectory(org.apache.spark.sql.SparkSession spark,
                                                                                    String path,
                                                                                    int minPartitions)

Returns an RDD of bundles loaded from the given path.

Parameters:

spark - the spark session

path - a path to a directory of FHIR Bundles

minPartitions - a suggested value for the minimal number of partitions

Returns:

an RDD of FHIR Bundles

fromJson

public org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> fromJson(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> jsonBundles,
                                                                           String column)

Returns an RDD of bundles loaded from the given dataset that has JSON-encoded bundles in the given column.

Parameters:

jsonBundles - a dataset of JSON-encoded bundles

column - the column in which the JSON bundle is stored

Returns:

an RDD of FHIR Bundles

fromJson

public org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> fromJson(org.apache.spark.sql.Dataset<String> jsonBundles)

Returns an RDD of bundles loaded from the given dataset of JSON-encoded bundles.

Parameters:

jsonBundles - a dataset of JSON-encoded bundles

Returns:

an RDD of FHIR Bundles

fromXml

public org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> fromXml(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> xmlBundles,
                                                                          String column)

Returns an RDD of bundles loaded from the given dataset that has XML-encoded bundles in the given column.

Parameters:

xmlBundles - a dataset of XML-encoded bundles

column - the column in which the XML bundle is stored

Returns:

an RDD of FHIR Bundles

fromXml

public org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> fromXml(org.apache.spark.sql.Dataset<String> xmlBundles)

Returns an RDD of bundles loaded from the given dataset of XML-encoded bundles.

Parameters:

xmlBundles - a dataset of XML-encoded bundles

Returns:

an RDD of FHIR Bundles

extractEntry

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> extractEntry(org.apache.spark.sql.SparkSession spark,
                                                                           org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles,
                                                                           Class resourceClass)

Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type.

Parameters:

spark - the spark session

bundles - an RDD of FHIR Bundles

resourceClass - the type of resource to extract.

Returns:

a dataset of the given resource

extractEntry

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> extractEntry(org.apache.spark.sql.SparkSession spark,
                                                                           org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles,
                                                                           String resourceTypeUrl)

Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type.

Parameters:

spark - the spark session

bundles - an RDD of FHIR Bundles

resourceTypeUrl - the URL identifying the FHIR resource type or profile.

Returns:

a dataset of the given resource

extractEntry

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> extractEntry(org.apache.spark.sql.SparkSession spark,
                                                                           org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles,
                                                                           Class resourceClass,
                                                                           List<Class> containedClasses)

Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type, including any declared resources contained to the parent resource.

Parameters:

spark - the spark session

bundles - the RDD of FHIR Bundles

resourceClass - the type of resource to extract.

containedClasses - the List of type of the resources contained to the parent resource.

Returns:

a dataset of the given resource

extractEntry

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> extractEntry(org.apache.spark.sql.SparkSession spark,
                                                                           org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles,
                                                                           String resourceTypeUrl,
                                                                           List<String> containedClassesUrls)

Extracts the given resource type from the RDD of bundles and returns it as a Dataset of that type, including any declared resources contained to the parent resource.

Parameters:

spark - the spark session

bundles - the RDD of FHIR Bundles

resourceTypeUrl - the url of the resource

containedClassesUrls - the list of urls of the resources contained to the parent resource

Returns:

a dataset of the given resource

saveAsDatabase

public void saveAsDatabase(org.apache.spark.sql.SparkSession spark,
                           org.apache.spark.api.java.JavaRDD<Bundles.BundleContainer> bundles,
                           String database,
                           String... resourceNames)

Saves an RDD of bundles as a database, where each table has the resource name. This offers a simple way to load and query bundles in a system, although users with more sophisticated ETL operations may want to explicitly write different entities.

Note this will access the given RDD of bundles once per resource name, so consumers with enough memory should consider calling JavaRDD.cache() so that RDD is not recomputed for each.

Parameters:

spark - the spark session

bundles - an RDD of FHIR Bundles

database - the name of the database to write to

resourceNames - names of resources to be extracted from the bundle and written