Thanks Bogdan for outlining the diagnosis steps. I have actually manged to resolve this issue. It's because one of the columns I passed in as prediction data does not comply with Datarobot requirement. The data drift is working now after I corrected it.
One suggestion for future deveopment is whethere there can be more meaningful error message returned from API calls. My API calls are sent to DataRobot without error, so I was asuuming everything I passed in was correct.
Speaking of the agent log, it's in a special fs_spool format which is not very friendly to be read.