Update README.md: Ajay's update for the Kafka / S3 Integration

README.md

@@ -4,4 +4,36 @@
 
 * OpenAQ: https://openaq.org/#/?_k=pt0eqe
 
-* OpenAQ API Docs: https://docs.openaq.org/
\ No newline at end of file
+* OpenAQ API Docs: https://docs.openaq.org/
+* 
+
+Kafka Integration (openaq s3 pull --> kafka topic push):
+
+Changes made to default kafka and broker settings.
+
+Broker Configs($KAFKA_HOME/config/server.properties) --> Add the below 2 properties. This is required for the large messages to pass through
+	• replica.fetch.max.bytes=60000000
+	• message.max.bytes=60000000
+
+I have 2 installation's:
+	1) On my mac its running as a background service. The service was installed using the homebrew commands.
+		a. https://medium.com/@Ankitthakur/apache-kafka-installation-on-mac-using-homebrew-a367cdefd273
+		b. This is the fastest and easiest way to start kafka service.
+		c. THE DEFAULT MECHANISM DOESN’T TAKE ANY CHANGES MADE TO THE PROPERTIES FILE, HENCE
+			i. Stop the brew services : brew services stop kafka & brew services stop zookeeper
+			ii. Change the properties under: /usr/local/etc/kafka/kafka.properties
+				• replica.fetch.max.bytes=60000000
+				• message.max.bytes=60000000
+			iii. Once saved and quit, do below in 2 separate terminals. These will then run in foreground.
+				1) zookeeper-service-start /usr/local/etc/kafka/zookeeper.properties
+				2) Kafka-server-start /usr/local/etc/kafka/server.properties
+	2) On my windows machine I followed similar steps and downloaded the latest version of Kafka.
+		a. NOTE: Kafka has internal zookeeper installation. Only use that since standalone different version creates problems.
+		b. On windows all the bat files are under : bin/windows.
+		c. All the steps required to run on windows is same as above.
+
+Application: Spring-Boot (OpenAQMainApplication.java)
+	1) There are settings I have hardcoded in java files which needs to be changed as per your configurations.
+	2) For your aws_access_key_id and aws_secret_access_key, these need go to C:\Users\kajay\.aws\credentials
+	3) Rest your local kafka IP address changes and configuration should be straight forward. An topic has to be created on your kafka for openaq-stream-in-data-topic.
+	4) S3OpenAQBucketReader -> this has the actual code for s3 read and kafka write. --> it works :)