Merge branch 'main' of https://gitlab.engr.illinois.edu/maxes2/ece-445-distributed-species-tracker

notebooks/Jonathan/README.md

@@ -3,7 +3,25 @@
 - [Jonathan Notebook](#jonathan-notebook)
 - [2023-01-31 - General Meeting Updates](#2022-01-31---general-meeting-updates)
 - [2023-02-02 - Initial Block Diagram](#2022-02-02---initial-block-diagram)
-- [2023-02-06 - Researching Power Submodule and Discussion with Jason](#2021-02-06---researching-power-submodule-and-discussion-with-jason)
+- [2023-02-06 - Researching Power Submodule and Discussion with Jason](#2023-02-06---researching-power-submodule-and-discussion-with-jason)
+- [2023-02-13 - More Power Management Discussion](#2023-02-13---More-Power-Management-Discussion)
+- [2023-02-15 - Hardware Design Discussion](#2023-02-15---Hardware-Design-Discussion)
+- [2023-02-20 - PIR Sensor](#2023-02-20---PIR-Sensor)
+- [2023-03-03 - PCB Design](#2023-03-03---PCB-Design)
+- [2023-03-04 - PCB Design](#2023-03-04---PCB-Design)
+- [2023-03-05 - PCB Design](#2023-03-05---PCB-Design)
+- [2023-03-06 - PCB Design](#2023-03-06---PCB-Design)
+- [2023-03-08 - PCB Order and OpenCV with Jetson](#2023-03-08---PCB-Order-and-OpenCV-with-Jetson)
+- [2023-03-21 - Initial CAD Design](#2023-03-21---Initial-CAD-Design)
+- [2023-03-26 - Tests and Verifications for Battery Charging Circuit](#2023-03-26---Tests-and-Verifications-for-Battery-Charging-Circuit)
+- [2023-04-01 - Soldering PCB Board](#2023-04-01---Soldering-PCB-Board)
+- [2023-04-03 - Flashing Attempt Via UART with Breakout Board](#2023-04-03---Flashing-Attempt-Via-UART-with-Breakout-Board)
+- [2023-04-09 - Flashing Attempt Via UART with the PCB](#2023-04-09---Flashing-Attempt-Via-UART-with-the-PCB)
+- [2023-04-10 - Flashing Attempt Via JTAG](#2023-04-10---Flashing-Attempt-Via-JTAG)
+- [2023-04-11 - Solder More PCB Boards](#2023-04-11---Solder-More-PCB-Boards)
+- [2023-04-12 - Battery Management Circuit Tests](#2023-04-12---Battery-Management-Circuit-Tests)
+- [2023-04-17 - Solar Panel Tests](#2023-04-17---Solar-Panel-Tests)
+
 
 # 2023-01-31 - General Meeting Updates
 
@@ -75,4 +93,52 @@ Continued to work on the PCB design in KiCAD. Ryan and I spent the majority of t
 # 2023-03-06 - PCB Design
 
 Ryan realized an important factor that the power module design would be insufficient for supplying enough current at 3.3V for the MCU, RFM, and GPS modules. This was due to a miscalculation of the MCU's current draw. It regulates at 3.3V but has a dropout voltage of 0.65V at 0.5A of current draw.  We can't afford 
-that much dropout since we're supplying the regulator with 3.7V and would need about 0.8A of current draw at peak times. Ryan suggested a possible solution by having 2 regulators, one regulator will be feeding into the MCU, and the other regulator will feed into the Radio, GPS, and PIR sensor to support the current draw of all of the components. We made these modifications to the PCB and finished routing all of the traces. We then went through the PCB checklist and audit and tweaked layout and addressed DRC warnings. The gerber files were submitted onto PCBway and passed the audit. We have finished our first iteration of our PCB Design.
\ No newline at end of file
+that much dropout since we're supplying the regulator with 3.7V and would need about 0.8A of current draw at peak times. Ryan suggested a possible solution by having 2 regulators, one regulator will be feeding into the MCU, and the other regulator will feed into the Radio, GPS, and PIR sensor to support the current draw of all of the components. We made these modifications to the PCB and finished routing all of the traces. We then went through the PCB checklist and audit and tweaked layout and addressed DRC warnings. The gerber files were submitted onto PCBway and passed the audit. We have finished our first iteration of our PCB Design.
+
+# 2023-03-08 - PCB Order and OpenCV with Jetson
+
+Performed final checkup of the PCB Design and submitted gerber files to TA for the first PCB order. Met up with Ryan and Max to begin working with the Jetson Nano. After learning how to boot up the Jetson, we installed all of the necessary software to use OpenCV. We finally wrote a simple python script that can read directly from a USB webcam. This step was extremely important to start the image classification process
+
+# 2023-03-21 - Initial CAD Design
+
+Began initial iteration of CAD Design of the physical design of the system. The Jetson Nano is the largest component in our design at 69.6 mm x 45 mm. We designed the PCB to be exactly the same size so we essentially need a rectangular prism with cutout holes for the PIR sensor, camera, GPS antenna, and RFM antenna. We currently do not know the size of the camera so I made a rectangular prism 75 mm x 50 mm x 50 mm. We may need to raise the height extrusion to make more room for heat dissipation of the Jetson and miscellaneous wiring. I am worried about the heat dissipation because the heat sink on the Jetson is quite bulky and there should not be anything in contact with the sink to dissapate correctly. Need to find a balance between ample ventilation throughout the system and weatherproofing of the exterior.
+
+# 2023-03-26 - Tests and Verifications for Battery Charging Circuit
+
+The first test and verification is for the battery charging circuit. According to the complete charge cycle provided by the TP4056 datasheet, it takes approximately 1.25 hours to nearly fully charge a 1000mAh battery.
+
+![](Charge_Cycle.png)
+
+Using the formula and estimating that battery charging will result in a maximum of 40% losses we can estimate the time to fully charge the 10050mAh 3.7V battery with a charging voltage of 4.2V and charging current of 1A to be 14.07 hours. To test this, we will charge and discharge the battery 10 times. Using the LEDs on the circuit, we can determine when the  battery is fully charged. We will then take the average of the 10 tests and perform error analysis in comparison to the calculated charging time. We can also simulate the battery capacity in relation to the system by discharging the battery with a current draw of 2.5 A. Next, the voltage regulators and boost converters must be tested to see if it can support the current draw requirements of our system. Testing and verifying this includes supplying the chips with a steady voltage supply from a power supply and probing the output to verify 3.3V and 5V output respectively. We can also simulate a current load of 2.5A and ensure that the voltage does not drop below the voltage requirements. If these tests are verified, the power management system is capable of supporting the entire node system.
+
+
+# 2023-04-01 - Soldering PCB Board
+
+Worked with Ryan to solder the first iteration of the PCB Board. We discussed which components we wanted to use the oven for and which components to hand solder. We decided to use the stencil and solder paste for the finer pitched footprints and baked the board with the oven. It took several bakes to get the components aligned because of solder bridging and the solder paste drying out. The USB footprint proved to be very difficult to solder correctly because of its very fine pitched contact points and those points were under the usb encasing, making it very hard to fix with flux. We then tried flashing the ESP32 MCU via the USB data lines but the computer would not recognize the chip as a device. We probe the PCB with the multimeter and confirmed that the MCU was bring powered and drawing current. The USB was also being powered correctly, but probing the data lines showed that nothing was being sent to the MCU. After talking to another group that was using a similar ESP32 MCU, we decided to order a UART to USB bridge to try to flash the MCU in a different way.
+
+# 2023-04-03 - Flashing Attempt Via UART with Breakout Board
+
+Using the UART to USB bridge, we began trying to flash our ESP32 Breakout Board with a simple LED blinking program. We eventually got the blinking LED program to flash via UART bridge, but we encountered invalid packet protocols when we attempted to flash the board with more complex programs such as our networking program.
+
+# 2023-04-09 - Flashing Attempt Via UART with the PCB
+
+Met with Ryan to try to directly flash the MCU on the PCB using the UART bridge instead of the microUSB on the PCB. Decided to unsolder the MCU and LoRa module that we baked on the PCB previously and resolder a new ESP32 chip to eliminate the possibility of a faulty MCU due to the number of times we needed to bake the PCB the first time. I soldered the ESP32 and LoRA module by hand to prevent any issues occurring with the high temperatures that the oven needs to reach to achieve reflow temperature. After fixing bridging with flux and Jason showing us how to properly clean off the pads, we were sure that there were no soldering issues with the board. The computer was able to recognize a valid serial power with the UART bridge, but we were still unable to flash the board due to more inalid packet header errors. Ryan continued to try to debug the system and implemented a pull-up resister circuit to manually drive some of the logic headers high and hard code values. Despite these efforts, we were still unable to flash the MCU.
+
+# 2023-04-10 - Flashing Attempt Via JTAG
+
+Attempted another possible flashing solution using a JTAG debugger board. Met with Ryan and Jason to try this method but ultimately did not work either. With this, Jason suggested that we move on from our attempts with flashing the MCU on the PCB to working with what we have on the breakout boards.
+
+# 2023-04-11 - Solder More PCB Boards
+
+Worked with Ryan to solder all of the components on 2 more PCB boards. Worked extensively to make sure that nothing was bridging and soldered the majority of the components by hand to prevent any issues occurring with the high reflow temperature point of the oven. The boost converter circuit IC proved to be impossible to solder even with the oven because the contact points were so fine pitched (even more than the microUSB) and laid under the IC making it impossible to fix any bridging issues with flux. We decided to move on from the boost converter circuit and power the jetson externally. We also tried flashing the 2 new PCBs and were still unsuccessful. However, we ran some initial tests of the battery management circuit by powering via power supply (3.7V) and probing contact points with the multimeter. The tests were promising as the voltage and circuit readings were as expected.
+
+# 2023-04-12 - Battery Management Circuit Tests
+
+Worked with Ryan to verify more of the battery management circuit tests. With the 3.7V LiPo batteries, we were successfully able to charge the LiPo battery via external power supply. The batteries charge at 1A of current which was programmed via a 1.2k resistor within the TP4056 schematic. The status LEDs also lit up correctly to signal whether the battery was currently charging or fully charged. The battery supplies 3.7V which is regulated to 3.3V to power the rest of the circuit. The next step will be testing the charging circuit with the solar panel rather than an external power supply.
+
+Discussed possible UI options of the website with Ryan and Max to display the published dta of the network. Ryan and Max were able to set up a simple server from the ESP32.
+
+# 2023-04-17 - Solar Panel Tests
+
+Ryan helped me setup the solar panels with the battery charging circuit. The solar panels are rated to 5.5V which should be sufficient to charge the batteries as they need 4.2V to charge the 3.7V batteries. Ryan was able to confirm that the batteries charge by shining a flashlight at the solar panels to generate over 5V. Ryan was also able to solder the solar panel and battery connections and ensure that one node's control system could be powered entirely from the solar panel setup. While the battery has a lower mAh capacity than we initially anticipated, we were able to show in demo purposes that our system can be sustained for an extended period of time with the battery constantly charging through the solar panel and discharging to power the circuit.
+

notebooks/Max/max_notebook.md

@@ -53,4 +53,32 @@ I have all three Jetson Nanos from Gruev, and yesterday and today I am working o
  - Board #2 lights up but does not show up on vga to hdmi on lab computers
  - Board #3 lights up but does not show up on vga to hdmi on lab computers
 
-Now board #2 lights up but turns off when the SD card is put in.
\ No newline at end of file
+Now board #2 lights up but turns off when the SD card is put in.
+
+3/20/2023:
+
+I managed to get the Jetsons working and today I am trying to get PyTorch working. It requires very specific packages of torch and torchvision, which I will download today. If this does not work, then fortunately the Jetson provides many other APIs for object detection, but since this is the API that I am the most familiar with I think it is the best course of action.
+
+3/21:
+
+After finding the correct packages, I have installed torch with Cuda, but every time I allocate any size tensor to Cuda, then the system adds 1M of memory, which slows down the computer and doesn't leave enough room for a model. So I think that I will either need to use their docker image which is even more specific to the Nano or I will have to use a higher-level wrapper. I will follow this course to see how they recommend doing it: https://courses.nvidia.com/courses/course-v1:DLI+S-RX-02+V2/.
+
+3/24:
+
+After doing the first few lessons of the course, I managed to set the swap memory to 4 MB by just following this video: https://www.youtube.com/watch?v=uvU8AXY1170, and I managed to run object detection at a very slow rate. When I use the docker image from the course, then once again, the system adds far too much memory than needed and the system slows down and there is not enough room for the model. But this method does actually work so if we cannot get any other setup working then this should be sufficient for at least some functionality.
+
+3/25:
+
+I have finally solved this bug and gotten a model loaded onto the system. To solve, I just built jetson inference from source (https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md), and modified the detection script that they provide. This solution did not even require PyTorch at all, and I believe that this software is specially created to run on the Jetson, making it an even better solution than what I tried before. Now our system can do object detection, and I will move onto using the PIO pins.
+
+3/26:
+
+I have gotten simple PIO input and output working by changing the permission levels of the PIO pins and using the provided examples in the Jetson directory. The interface is the same as a Raspberry Pi and is very simple to get set up.
+
+3/28:
+
+We are working on connecting the Jetson to the MCU through the PIO pins and we have found that there is a minimum of 1s needed for the Jetson to recognize an input signal from the MCU and vice versa. This will become a bottleneck in our final system, because we will not be able to capture and process the images as fast as we had hoped. We will have to modify our requirement of capturing and processing in 100 ms to 2 s to make up for this. In addition, we have found that SPI will not work as an interface between the Jetson and the MCU, because the Jetson and MCU are oth configured to be in master mode in SPI. Thus, we have decided to create our own protocol where the MCU uses one wire to tell the Jetson to take and process an image and the Jetson uses two wires to tell the MCU whether or not a human was present in the photograph.
+
+4/15:
+
+We have finished all parts of the project and we are running several demos. Our demo consists of setting up the three nodes and then triggering the different sensors and sending positive and negative classifications as needed. We will also run the experiments for model classification and accuracy today as well.
\ No newline at end of file

notebooks/Ryan/Ryan_Notebook

 Ryan Day
 1/28/2023
 
-I ordered an ESP32-S3-DevkitC-1U-N8R8 development board from Mouser in order to start early stage testing and development.  Today the dev board arrived.  We met as a team in order to start playing around with it.  Our time was spent installing and configuring the ESP-IDF framework and trying to hook up Jon's laptop to the board.  We faced issues with successfully flashing code onto the board.  The code we were trying to flash was some given example code in the esp-idf downloaded directory that is intended to blink an on-board LED.  
+I ordered an ESP32-S3-DevkitC-1U-N8R8 development board from Mouser in order to start early stage testing and development.  Today the dev board 
+arrived.  We met as a team in order to start playing around with it.  Our time was spent installing and configuring the ESP-IDF framework and 
+trying to hook up Jon's laptop to the board.  We faced issues with successfully flashing code onto the board.  The code we were trying to flash was some given example code in the esp-idf downloaded directory that is intended to blink an on-board LED.  
 
     -- Ryan Day
 
@@ -10,13 +12,21 @@ I ordered an ESP32-S3-DevkitC-1U-N8R8 development board from Mouser in order to
 Ryan Day
 1/31/2023
 
-This morning, I spent time on the dev board myself, picking up where we left off on Saturday.  I was able to follow the installation and configuration steps myself and was able to successfully flash software on a board.  I discovered that using the USB-UART bridge for flashing is necessary without an external programmer or a JTAG debugger.  Luckily, a USB-UART bridge comes with the dev board we ordered.  From there, flashing was simple.  
+This morning, I spent time on the dev board myself, picking up where we left off on Saturday.  I was able to follow the installation and 
+configuration steps myself and was able to successfully flash software on a board.  I discovered that using the USB-UART bridge for flashing is 
+necessary without an external programmer or a JTAG debugger.  Luckily, a USB-UART bridge comes with the dev board we ordered.  From there, 
+flashing was simple.  
 
-Later in the day, we met up as a team and worked on flashing slightly modified software that configures an output GPIO pin in order to blink an LED that we placed on a breadboard. We then began working on a custom test case/program that will take as input a voltage from a battery being toggled by a photoresistor via a GPIO pin and then correspondingly command an LED to be blinked via an output GPIO pin. The program built and flashed with no issues but the LED did not light up.  We were interrupted by the start of lecture so we will pick up on this another day. 
+Later in the day, we met up as a team and worked on flashing slightly modified software that configures an output GPIO pin in order to blink an 
+LED that we placed on a breadboard. We then began working on a custom test case/program that will take as input a voltage from a battery being 
+toggled by a photoresistor via a GPIO pin and then correspondingly command an LED to be blinked via an output GPIO pin. The program built and 
+flashed with no issues but the LED did not light up.  We were interrupted by the start of lecture so we will pick up on this another day. 
 
-Simultaneously, Max was working on looking into image classifier models we could use for our project.  We will likely use C++ to process our images.
+Simultaneously, Max was working on looking into image classifier models we could use for our project.  We will likely use C++ to process our 
+images.
 
-In addition, I found the espressif/esp32-camera github folder that hosts drivers compatible for our ESP32-S3 and a variety of camera sensors.  I will read into these in more detail later. 
+In addition, I found the espressif/esp32-camera github folder that hosts drivers compatible for our ESP32-S3 and a variety of camera sensors.  I 
+will read into these in more detail later. 
 
     -- Ryan Day
 
@@ -51,10 +61,9 @@ Ryan Day
 2/13/2023
 
 Today I spent some time working with the RFM95W more.  I went to the lab and used the oscilliscope to probe the SCK and CS lines to figure 
-out the SPI issue from yesterday.  I discovered that the SCK line wasn't behaving properly. I realized that there were 2 issues.  The first 
-issue being that I was using the wrong pin on the devkit for the SCK.  I switched this to pin 12.  The next issue was that the default sub
+out the SPI issue from yesterday.  I discovered that the SCK line wasn't behaving properly. I realized that there were 2 issues.  The first issue being that I was using the wrong pin on the devkit for the SCK.  I switched this to pin 12.  The next issue was that the default sub
  class used in the RadioHead driver as the SPI interface is a hardware SPI class that doesn't initialize any pins for SCK, MISO, and MOSI by 
- default.  I realized that another potential SPI subclass had a setPins function that I could pass values to configure these pins.  I used 
+ default. I realized that another potential SPI subclass had a setPins function that I could pass values to configure these pins.  I used 
  an instance of this class with the RF95 constructor instead and was able to get through the SPI initialization process.  At least right 
  now, I think it's working.  
 
@@ -67,7 +76,10 @@ This evening, I went to the lab to try to get two RFM95W modules communicating.
 receiver module to run on the ESP32-S3.  The transmitter packages a message into a buffer, sends the header and payload over SPI to the
  RFM95W, and then transmits it and waits for a response.  The receiver waits for an interrupt to trigger a handler that validates a receive
 buffer and puts the transceiver into Rx mode before copying the contents of the receive buffer into it's own buffer for reading.  I used the 
-Arduino library to create separate threads for each of these scripts since we have one devboard.  I kept running into an issue where the transceiver would send the message fine but the receiver wouldn't even have it's interrupt handler called - a watchdog timer was being triggered first, causing a boot loop.  I used debugging print statements throughout the code and the driver to rule out issues with the rx buffer validation, the funciton that waits for availability, and the interrupt handler logic itself.  
+Arduino library to create separate threads for each of these scripts since we have one devboard.  I kept running into an issue where the 
+transceiver would send the message fine but the receiver wouldn't even have it's interrupt handler called - a watchdog timer was being triggered 
+first, causing a boot loop.  I used debugging print statements throughout the code and the driver to rule out issues with the rx buffer 
+validation, the funciton that waits for availability, and the interrupt handler logic itself.  
 
 Ryan Day
 
@@ -249,6 +261,7 @@ Today, Jason helped me try to flash the MCU on the PCB using a JTAG debugger boa
 
 Ryan Day
 
+
 Ryan Day
 4/11/2023
 
@@ -258,6 +271,7 @@ encouraging.  Tomorrow, some 3.7V lipo batteries that I ordered will be coming i
 
 Ryan Day
 
+
 Ryan Day
 4/12/2023
 
@@ -302,3 +316,44 @@ by the IR sensor).  We also made sure that the battery management circuit was ca
 
 Ryan Day
 
+
+Ryan Day
+4/17/2023
+
+Today I spent time trying to debug the issue with flashing the MCU.  I used an oscilloscope to measure the voltage level on the power pins and 
+the strapping pins to make sure they were staying at a constant level and not dipping below a threshold and triggering a reset (this was done 
+by using a trigger on the oscilliscope).  All of the values seemed correct.  I then turned my attention to the ESD chip and resoldered it to make 
+sure that there was no bridging.  I verified that there was proper power on the ESD chip but couldn't see good results when probing the data 
+lines.  I would expect to see something like a square wave but was getting values that looked like floating logic levels.
+
+Ryan Day
+
+
+Ryan Day
+4/19/2023
+
+I tried to bypass the ESD chip entirely using jumper wires and just connecting straight from the USB port to the MCU data pins.  However, I 
+struggled to get good soldering done for this.  Eventually I was able to solder wires in a way where I was fairly confident there was no bridging.
+Unfortunately, there was still a connection issue and my computer couldn't recognize the ESP32.  I am curious as to whether there is simply 
+an issue with the USB port and that they were all improperly soldered or something. Both the USB and the MCU would need to be sending data 
+reliably to establish any connection and initiate a session. 
+
+Today I also conducted a range test between two LoRa nodes.  I set the SF to 7, the coding rate to 4/5 and measured a max distance of 0.35km.  
+The RSSI value for this distance was -127dBm.  This is definitely below the noise floor and pushing the boundaries of what the LoRa module can 
+demodulate.  Although I was able to maintain a direct line of sight between the two antennas, unfortunately, I was unable to find a great place 
+where the Fresnel zone is unimpeded by objects.  This has a negative effect on the strenght of the signal and certainly depletes the range of 
+distance in which a message can be sent. Later, I will test max distances and RSSI values for other combinations of spreading factor and coding 
+rate.  
+
+Ryan Day
+
+
+Ryan Day
+4/21/2023
+
+Today Max and I ran through a mock demo together and took a video to send to Hanyin.  We demonstrated the functionality of our battery management 
+circuit, the image classifier, and the networking module by running through some classifications on each node and verifying that the results were 
+displayed on a server run by an arbitrary node.  
+
+Ryan Day
+