When you know that a table exists in Hive / Impala, either because you created it, or you are able to query the table and view it’s data through UI or CLI, and also, you are able to query the table through pyspark or spark-shell, but then when you access the same table in spark-submit command, the application throws error ‘Table not found’, then the most likely cause is that
You are using sqlContext in your SparkContext [ SqlContext(sc)] rather than using HiveContext [ HiveContext(sc) ].
Change your code from SqlContext to HiveContext, and your code would be able to find the table again !!
Senthamizh Samikkanu's Code Snippets - Python, Perl, SQL, Tableau etc 