cancel
Showing results for 
Search instead for 
Did you mean: 

Loading multiple datasets in a single project

Loading multiple datasets in a single project

I need to load multiple datasets in a single project. But the project allows to load only 1 time.

My requirement is to load different files, apply different processing to them and bring them to common format and then row-append them.

How do I do this under a single project in Paxata?
Labels (1)
6 Replies

Hello,

In order to do this, you would need to create Paxata projects for each one of your datasets to bring them to the desired common format. Once this is done in a project, you could publish these projects to the Paxata Library and then create a fresh Paxata project to row append all the desired projects.

I hope this makes sense to you! Let me know if you need any further clarifications!


Another approach would be to append the files in a single project first. Then expand each individual Append step and do the specific transformations to that dataset. By the way, this technique can also be done when doing a Lookup too.
Image: https://us.v-cdn.net/6030933/uploads/editor/ck/jvjctdfqu2we.png

@Kumar Jayaram
I don't see any "import" operation as you have shown in screenshot... may be, this is a new feature?

In my case, the schema of the files itself is very different. I cant append before I do transformations on each single dataset.... 

I was able to find the "expand Append option" after which I can see the "import" step. This is nice.
But what I am missing is that once I reach this step, I am unable to do schema transformation based on Lookups. That step alone is disabled while I am at the import step. All other options are available....Argh Argh! Any solutions

Hi @MagmaMan, it is not possible to do add a lookup/append step as a sub-step before a lookup/append. This is a product design decision. You can add the lookup step after the append is performed. Another option is to do your lookup in a separate project, publish the output (answerset), and append the answerset to your main project. Does that help?

@mikeblanko
Thanks..... Ah... I can understand if Append is not allowed again...as it leads to a recursive trap...
but why not lookup?
btw.. I already accomplished this using a separate projects.... possibly tag all these projects to make them traceable as well... But, still if all are stored in a single project, it will be really good.