REST API Primer

[ebarre1]

What is the Paxata REST API?
Taken straight from our REST API documentation:

To provide programmatic access to the Paxata application, the Paxata application publishes a RESTful web service called the Paxata REST API. The Paxata REST API requires user-level authentication with a valid user account on the server. The Paxata REST API enforces permissions in the same way as the web application user interface. Additionally, for programmatic access the user account must have the AccessRestApi permission. By default, the RemoteAccess and SuperUser roles contain the AccessRestApi permission.

So basically, you can access all the functionalities of Paxata through code using Python, Java or any other modern language.
All access and requests are restricted and submitted to the same auditable and governable processes that our web interface goes through.

Hmm, ok but I don't need it to use Paxata, right?
 Correct, think of the REST API as being able to customize and integrate Paxata within your current workflow. However, you might be asked to provide a REST API token in order to authenticate, this token can only be generated by you and can only be seen once but can be re-generated and deleted at will, adding an extra layer of security.

What type of operations can be performed using the REST API?

  As mentioned earlier, the REST API provides access to Paxata programmatically, it does so by exposing URLs (called endpoints) to different types of actions, defined by different verbs. GET(), retrieves a list of something, POST(), creates one of those something, PUT(), updates it, DELETE() removes it.

  An endpoint can support any number of those verbs, and responses sent from Paxata will also change based on the information provided to the endpoint. For example, requesting a GET() on the library endpoint will return a list of all the library items that you have access to, while doing the same GET() and passing a library ID will return information about this specific dataset/answerset.

What is an example of integration?

  A lot of the organizations we work with utilize some type of enterprise scheduler, be it a CRON job, an Autosys or Control-M scripts. Paxata can be called as part of these tool processes and perform actions based on the scripts workflow, I.E. Import my sales data into Paxata to prep it for my projection model as soon as the previous process completed successfully, or, execute the data quality metrics project for the BI Dashboard.

  But since the REST API is accessible through code, Paxata can also be integrated with any in-house development efforts.

What is an example of a utility?

  The early utilities were built as tools to facilitate our work, such as documenting projects as artifacts for the customer,
other utilities have come from requests from our customers to have a tighter integration with their process.

Where can I get more information on the REST API?
  Our documentation site has all that you need to interact with our REST API, if you have questions on how to get started then contact our Customer Services team.

[abhijeetmkhambe]

Hi Bill (bstephens),
Thanks for your reply.

Please find below my answers-

1. "Target host is not specified":  "https://paxataurl:portnumber" is not a valid host/port combination. You need to substitute your actual Paxata server name and port.  - Yes. I replaced my actual server url and port number with some random text while posting my question here.
2. The "-F" entry is specific to curl. This won't work from Java. - Thanks. I removed -F
3. Your SQL query string needs to be URL Encoded - Thanks. I did this.

I did get the error  The method encodeBase64(byte[]) is undefined for the type Base64 when using 
byte[] encodedAuth = Base64.encodeBase64(auth.getBytes(Charset.forName("UTF-8")));

As an alternative, I used below method- 
byte[] encodedAuth = Base64.getEncoder().encode(auth.getBytes(Charset.forName("UTF-8")));



Here is my complete java code.

@RestController

public class Test {

@RequestMapping(value = "/query", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)

@ApiOperation(value = "This API posts a new dataset in paxata against a query")

@ApiResponses(value = { @ApiResponse(code = 200, message = "Successfully added new dataset details in paxata"),

@ApiResponse(code = 404, message = "Unable to add new dataset details in paxata"),

@ApiResponse(code = 500, message = "Internal server error") })

@ResponseBody

public InputStream DatasourceImport(String name, String dataSourceId, String query) throws ClientProtocolException,Exception {

dataSourceId="1234567890";

query = "select * from empschema.employee_table";

name = "Testing_dataset";

String encodedQuery = URLEncoder.encode(query, StandardCharsets.UTF_8.toString());

String url = "https://paxataurl:portnumber/rest/datasource/imports/" + dataSourceId + "?name=" + name + "&query=" + encodedQuery;

HttpClient client = HttpClientBuilder.create().build();

HttpPost postRequest = new HttpPost(url);

postRequest.addHeader("Content-Type", "application/json");

String auth = ":12345678";

byte[] encodedAuth = Base64.getEncoder().encode(auth.getBytes(Charset.forName("UTF-8")));

String authHeader = "Basic " + new String(encodedAuth);

postRequest.addHeader(HttpHeaders.AUTHORIZATION, authHeader);

HttpResponse response = client.execute(postRequest);

HttpEntity entity = new BufferedHttpEntity(response.getEntity());

return entity.getContent();

}

}

Now, I am getting error as below-
DefaultHandlerExceptionResolver : Resolved [org.springframework.web.HttpMediaTypeNotSupportedException: Content type '' not supported]

Even after setting Content-Type error is not resolved.

Kindly help.

Regards,
Abhijeet

[bstephens]

@abhijeetmkhambe

You are using "java.util.Base64"  I was using "org.apache.commons.codec.binary.Base64".

The latest SpringFramework error is out of scope for answering here as it's not a Paxata specific question / issue. 

Thanks,
Bill 

[abhijeetmkhambe]

Hi bstephens,
Thanks. I did the change you suggested. We are still getting SSL handshake related error. We are working on the same. Once my code works, I will update here.

Regards,
Abhijeet

[ebarre1]

Hello @abhijeetmkhambe

I have attached a sample file that imports from various sources (including form a query). I suspect that the SSL handshake error you are experiencing is caused by a self-signed certificate on your environment. The executeRequest on the sample file (line 153) should get you what you need.

P.S. I had to rename the file to .txt so that I could upload it, change it back to .java.

[abhijeetmkhambe]

Hello @bstephens
I was able to resolve ssl handshakre error and create dataset successfully through java code. Not sure why we are getting below error at java side even though operation is successful.

WARN 15720 --- [nio-8080-exec-1] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.http.converter.HttpMessageNotWritableException: No converter found for return value of type: class java.io.ByteArrayInputStream]

 Do we need to change return type of our request? Kindly guide me.

Thanks,
Abhijeet



@ebarre
I went through the file you shared. 
At line number 153 
private static HttpResponse executeRequest(HashMap<String, String> reqOpt, String verb, String endpoint, String opt, File f)
one of the input parameters is File type. But we are fetching the query from database and executing it against paxata. Kindly guide if I am missing something here.


Thanks,
Abhijeet

[ebarre1]

@abhijeetmkhambe

Nope, you are not missing anything :smile:

When calling this method just pass null for the last argument. If you notice in the code  (line 186 -189):

        if (f != null) {

            String importFileName = f.getName();

            builder.addBinaryBody("data", f, ContentType.APPLICATION_OCTET_STREAM, importFileName);

        }

 I am only adding the data object to my request if the last argument is not null. Look at line 47 for a usage example.

This code is only a sample so it might not be exactly what you need but hopefully, it can be useful for you to develop a solution based on your own requirements.

Hope this helps, let me know if you have other questions

[abhijeetmkhambe]

Thanks everyone for your inputs.
I would like to know, if we are creating httpclient and executing paxata API's, we get output as HttpResponse. Is there any way, I can get JSON output which will give me all the details of dataset created?

Regards,
Abhijeet

[bstephens]

@abhijeetmkhambe

The original code that I posted on Oct 11 gets you a String containing the response (HttpEntity) content. If you need to process that String as JSON, you'll need to use some JSON library to convert this string into a JSONObject/JSONArray.

Some obvious candidates: 

• org.json.JSONArray  for Array responses
• org.json.JSONObject for single Objects (datasource import returns a string representation of this)

Here is a link to a StackOverflow question that shows a few options. 
https://stackoverflow.com/questions/10804466/how-to-convert-httpentity-into-json 

Thanks,
Bill

[abhijeetmkhambe]

@bstephens
Thanks. It worked with some amendments.

Regards,
Abhijeet

[Noahkays]

Are their any PostMan collections available to help me get a jump start on the Paxata API's that are available?