Author: Naveen P.N

12+ years of experience in IT with vast experience in executing complex projects using Java, Micro Services , Big Data and Cloud Platforms. I found NPN Training Pvt Ltd a India based startup to provide high quality training for IT professionals. I have trained more than 3000+ IT professionals and helped them to succeed in their career in different technologies. I am very passionate about Technology and Training. I have spent 12 years at Siemens, Yahoo, Amazon and Cisco, developing and managing technology.

Handling “Hive Metastore not working – Syntax error ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1”

Problem Description While dropping a hive table the following exception encountered in hive shell. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘OPTION SQL_SELECT_LIMIT=DEFAULT’ at line 1) Hive version – 2.3.3 mysql-java-connector version –...

Apache Spark : Loading CSV file Using Custom Timestamp Format

In this blog post, we will see how to load csv which contains timestamp as one of the column. Creating DataFrame from CSV file If you see the below data set it contains 2 columns event-name and event-date.The event-date column is a timestamp with following format “DD-MM-YYYY HH MM SS“. EVENT_ID,EVENT_DATE AUTUMN-L001,20-01-2019 15 40 23 AUTUMN-L002,21-01-2019 01 20 12 AUTUMN-L003,22-01-2019...