IBM Analytics Ideas

Welcome to the idea forum for IBM Analytics Clients! 

 

IBM Employees:

The correct URL for entering your ideas is https://hybridcloudunit-internal.ideas.aha.io

 

Clients:

Our team welcomes any feedback  and suggestions you have for improving our offerings / products!  This forum allows us to connect your offering / product improvement ideas with IBM product and engineering teams.

 

If you have not registered on this portal please click on the following link and register.  To complete registration you will need to open the email you will receive from Aha to confirm your identity.  https://ibm.biz/AnalyticsIdeasPortalRegister

 

Allow recursive directory read as hive does.

Hive developer in our organization prefers to create the multiple subdirectories to store the data inside parent directory mentioned in DDL for unpartitioned tables but bigsql only looks up for ORC files in parent directory. It doesnt even bother checking subdirectories. Hive instead read all the sub directories and displays the result.

 

Example - create table t1 ( c1 int) location hdfs:///tmp stored as orc;

 

Now the developers have created few sub folders inside the parent /tmp directory in hdfs to store the data.

 

/tmp/part/00001/00000.orc

/tmp/part/00002/00000.orc

 

In this scenario, bigsql only tries to read the data from /tmp directory and display an empty table because it cannot read the data recursively.

 

  • Guest
  • Jul 18 2018
  • Needs review
Role Summary Hadoop Administrator
  • Attach files

NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "anonymous@euprivacy.out" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions