Hi again amodi I know a community SME will give you some guidance here, but I also wanted to point you to this learning session from March which may be useful to you, if you haven't seen it already. AI in the Reail Industry
Thanks a lot for taking out your precious time to answer this question. It really helps.
Can you please direct me with the nuances I should keep in mind to do this that are not directly done using DR, like suggestions for pre-processing, feature engineering, maths involved (for maximization of profit by minimization of discount) etc
Also can you please guide me through any research papers and suggested python based example to get a complete perspective of this analysis.
II would always try to predict the quantity that is least affected by any business decisions. In this instance it would be demand, or sales quantity.
Once you have a good model that predicts your expected demand based on some relevant features (like product attributes, location, date, holidays, and of course price/promotions) you can use this model to start asking 'what if' type questions.
So what if I reduced the price, or what if I put this product in a 'buy one get one free' promotion, etc. The important part is that you should only try to use one lever at a time. If the promotion impacts the price, use the price only. That keeps it simple for the model.
Now you've got the model, you can indeed make use of the optimisation app, where you ask to maximise profit = (price - cost) * quantity, where quantity is now of course a function of price, whereas cost typically stays constant.