If your problem is simply that you have too much chlorine in your drinking water & it is not within your power to reduce the levels at source, then I would suggest you instal an activated carbon filter. This should knock out all of the chlorine.
You do, however then need to be careful as the chlorine that was doing a wonderful job protecting your water from infection is now gone, so I would advise that you use the water immediately after the activated carbon filtration.
Sufficient chlorine is required to meet the chlorine demand (to disinfect the contaminants present) and to leave a 'residual' chlorine that will give protection from possible future contamination. Typically the target residual is 0.2-0.5 mg/l (as specified by WHO). It sounds like you are over dosing the chlorine. To determine how much chlorine you require you will need to analyse the water for the chlorine residual. A simple colour comparison kit with reagents are normally easy to buy in most countries.
If you provide more information about this particular situation we can give you more help.
This thread is public, all members of KnowledgePoint can read this page.