Hadoop Job Completion Process
List HDFS to ensure nothing's there
[path]/hadoop fs -ls
Load the data
[path]/hadoop fs -put ~/Documents/DataFiles/StationData stationDataDir
List HDFS to ensure data has been loaded
[path]/hadoop fs -ls
Run the job
[path]/hadoop jar [path]InstructionalSolutions.jar AverageTempByStation stationDataDir avgTempbyStationOutDir 2
List HDFS to see what happened
[path]/hadoop fs -ls
Get Job Output
[path]/hadoop fs -get avgTempbyStationOutDir ~/Downloads/stationDataOutput
Cleanup HDFS
[path]/hadoop fs -rmr stationDataDir
[path]/hadoop fs -rmr avgTempbyStationOutDir
Where Silva Wants Your Input
~/Documents/DataFiles
Where Silva Wants Your Output
~/Downloads
Where Silva Wants Your Programs
~/bin/Hadoop-1.0.4
|