All posts by tchiko

How to keep a program running using spark streaming

I using Apache Spark on my Java project, the result of the program occurs only when I run my program, now I want my program to be always up and running using SparkStreaming.

My project is structured as follow :

  • Package.launch :

    public class App

    {

    public App(){
    
        new Launch();
    }
    public static void main( String[] args )
    {
        new App();
    }
    

    }

    public class Launch {

    Read read = new Read();
    Transform transform = new Transform();
    Write write = new Write();
    
    public Launch(){
    
        write.getWriter(
                        transform.getTransformer(
                                                    read.getReader())); 
    }   
    

    }

  • Package.read :

    public class Read {

    public Dataset getReader(){
        // Read from csv file then return dataset
        return ds;
    }
    

    }

  • Package.transform :

    public class Transform {

    public Dataset getTransformer(Dataset ds){
    
        //do transfomration on Dataset ds then return the final Dataset
    
        return ds;
    
    }
    

    }

  • Package.write :

    public class Write {

    public void getWriter(Dataset ds){
        // write the result on csv file 
    }
    

    }