Interactive Voice Response System With Raspberry Pi 4 & SIM800L

A low-literate audience can nevertheless have their voices heard and their questions answered by using an IVR system, as has been proven time and time again. However, achieving such aims in a development setting calls for a cheap system that welcomes input from various parties. RASP-IVR is an inexpensive IVR system that operates on a PI 4 and a local Global System for Mobile Communications modem. RASP-IVR was designed as an open-source, community-driven solution. It's unusual to find a customer-focused company that still uses human operators rather than an interactive voice response system. Credit card companies typically have IVR systems that can be used to make payments or file fraud reports. Airlines use elaborate IVR systems to schedule flights and check their current status. To facilitate medication refills, pharmacies implement interactive voice response systems. Furthermore, IVRs are widely used for forwarding calls to other extensions and providing directory assistance.

Enterprises of all sizes have adopted IVR technology due to its cost savings over using actual, flesh-and-blood staff. The number of callers who desire to speak with a human indicates an IVR system's success. As the percentage drops, the system becomes more efficient. Of course, some IVRs will never let you bypass the system and talk to a human being. However, that's frowned upon even by IVR advocates. As most people know, Raspberry Pi is a development board that packs a fair amount of processing power into a little package (about the size of a palm). When coupled with python's adaptability, this might lead to the creation of a wide variety of useful but diminutive devices. I recently had to construct an interactive voice response system using a Raspberry Pi, which essentially entails dialing phone numbers, sending messages, and receiving DTMF inputs from the recipient when they pick up the phone.

Where To Buy?
No.ComponentsDistributorLink To Buy
1Jumper WiresAmazonBuy Now
2Raspberry Pi 4AmazonBuy Now

Components

  • Pi 4

  • SIM800L GSM Module

  • 2G SIM Card (Airtel)

  • Aux Cable

  • LM2596 Buck Converter Module

  • USB to TTL Converter

  • 12V 2A Adapter

  • Perf Board

  • Berg Sticks

  • Connecting wires

  • Soldering Kit

Interactive voice response and its Operation

Computer and phone systems can be integrated into various ways, and IVR systems are one such way (CTI). Key tones are the most common way a phone can exchange data with a computer. Dual-tone multi-frequency frequency is what you get. A telephone's number pad emits a low and high-frequency tone. A "1" can be generated in a 697-Hz tone or a 1209-Hz tone, both of which are recognized by the PSTN as the numeral 1. For a computer to interpret the DTMF frequency produced by a phone, specialized hardware known as a telephony circuit or telephony board is required. Using a telephony board to connect a computer to a phone line and some low-cost IVR software, it is possible to create a rudimentary IVR system. The IVR program lets you record messages and menu items that callers can navigate using their phone keypads.

Speech-recognition software is a part of more sophisticated IVR systems, enabling callers to interact with computers via voice commands. The technology behind speech recognition is now advanced enough to comprehend things like full names and long series of digits. Text-to-speech software can be used on the receiving end to fully automate a firm's outgoing telephone calls. To better serve customers, computers may now generate personalized text to answer their inquiries, such as financial statements or flight times, and have an automatic voice read to them. Voice extendable mark-up language is a popular foundation for many cutting-edge interactive voice response systems today.

RASP-IVR: A Comprehensive Approach to Interactive Voice Response

The "RASP-IVR" moniker comes from the fact that it was created on the inexpensive Linux-based single-board computer known as a raspberry pi. The operating system is built on RASPBX, with several additions and modifications to improve compatibility for interactive voice response (IVR) use. Model 4 Raspberry Pi and GSM mobile phone modem make up the system. An Asterisk server is connected to the GSM modem via an interface. FreePBX is a web-based interface for managing an Asterisk server, allowing users to set up and modify IVR menus and VoIP settings. Using open-source libraries, we've written scripts to bring RASPBX's functionality for Voice over IP (VoIP), short messaging service, and localized SIP transfers to a broader.

Depending on the situation, the Asterisk domain server can route the incoming GSM call over a local or worldwide VoIP network. The VoIP communications can be directed to different extensions based on the client's dial input. They can be answered from any desktop or mobile device with a domestic SIP account and a SIP client like Zoiper. Furthermore, the VoIP conversation can be routed to third-party platforms like Twillio to use their cloud-based solutions like speech-to-text engines or cloud-based storage. In contrast, RASP-IVR requires no external resources, making it ideal for collaborative innovation or limited rollouts.

Why use GSM Module and not just the API to make phone calls?

The cost of using an API to make phone calls from a Raspberry Pi can add up quickly. You can use python and Raspberry Pi to make inexpensive phone calls using several different service providers, including Twilio. These solutions are fantastic; however, the price has two significant drawbacks. Almost wherever you go, you'll be charged by the minute, which goes for both outgoing and incoming communications. You might not think this is significant, but if you make more than a hundred calls daily, you'll soon realize how much it costs. The second issue is buying and managing a different cellphone number from which to make these calls. You cannot use an existing number because these are all made up, and depending on your area, you may have to pay more to get a number that works in your nation.

How can I make inexpensive phone calls with a Raspberry Pi 4?

After researching the rates of several API-based telephone conversation services, it became evident that the cheapest option is to use a Gsm technology like SIM800L with Pi to place calls through a network service provider like Airtel. To that aim, we'll connect a SIM800L module to a Raspberry Pi so we can send text messages and make phone calls automatically.

Which GSM module should we use with my Raspberry Pi 4 to do IVR calls?

As 5G and the IoT become increasingly popular, numerous innovative new GSM modules have been released that are compatible with 5G networks while using less energy. But I had the bright notion of employing a ubiquitous and inexpensive commodity to purchase at any nearby hobby store. So, we put the popular SIM800L and the Ai-thinking A9 through their paces. Even when just powered by my laptop's USB port, the artificial intelligent-thinker A9 Gsm modem proved completely reliable and responsive right from the get-go. However, the primary issue was that the in-line voice input interface did not operate very well, and the artificial intelligent Thinker A9 Gsm modem did not enable DTMF recognition. So, I switched to the SIM800L Global System for Mobile communications Modem, a power-hungry beast that, if fed enough juice, proved ideal for this task. There is no need for expensive external DTMF hardware because it has both in-line audio for playing automated voice and built-in DTMF detection. 

What kind of SIM card works with the SIM800L?

The SIM800L is only compatible with 2G networks, not 3G or 4G. Only Airtel and BSNL provide 2G services in this area. Therefore, get an Airtel SIM card or use one we already have. After purchasing a new Airtel subscriber identity module, card, I activated it on my cell phone and began using it with my SIM800L. 

A SIM800L-Based interactive voice response System's Typical Circuit Diagram

Complete schematics for a Raspberry Pi-based telephone and message system are provided below. It's clear how simple it is to draw connections between ideas.

Powering the SIM800L Module

The method by which the SIM800L module is supplied with electricity is crucial here. Within a voltage range of 3.7V to 4.2V, the SIM800L chip functioned. The optimum working voltage is close to 4V. For the SIM800L module, we needed to reduce the adapter's 12V 2A output to 4V; thus, we utilized an LM2596 Buck converter. Thicker, shorter wires are preferable when linking the buck component to the SIM800L so that the module can easily handle the high current. When powered, if the element detects improper power connections, it will reset and send invalid data through the serial port. You'll need a 12V 2A adapter to power the SIM800L module, and you should use bulky wires just to carry a lot of current without causing too much resistance. Connecting a fully loaded Lithium battery across the SIM800L module's Vcc and Ground pins should resolve any power difficulties.

SIM800L-to-Raspberry Pi Serial Data Transfer

Those unfamiliar with the jargon of electronics engineering know that "AT Commands" are what we use to talk to the SIM800L module. Calling, texting, and keypress detection are just a few of the many functions that may be accomplished with AT commands. This tutorial will use Py from RPi to transmit these AT instructions to the Global System for Mobile Communications SIM800L module. Connecting the SIM800L module's Rx and Tx pins to the Raspberry Pi's USB port required a universal serial bus to TTL converter.

Audio transmission from the Pi 4 to the SIM800L component

Through its MIC+ and microphone- pins, the SIM800L component supports mic input. Any audio information sent to these pins after the call has been placed will play on the call recipient's phone. In our scenario, we have to play recorded audio from the Rpi; however, these go pins are primarily designed to connect a microphone. To hear it on the GSM component, we must convert our audio output from the RPi through the 3.5mm port, known as line-out audio, to mic-level audio. You might construct a line-to-mic converter circuit to complete this task professionally. Still, I connected the two devices and set the volume level on the Raspberry Pi to barely two decibels. I had no trouble getting it to function this way.

I assembled the entire circuit on the perf board, ensuring the voltage connectors got enough lead to offer low-resistance contact. This is how my board appears after soldering. Since the LM2596 is an adjustable buck converter, ensure you use the onboard trim pot to set the voltage output to 4 Volts before using it. Anything above 4.2 volts can permanently damage the SIM800L component.

Simply turn on the circuit and attach the Cell phone card when you get here. Once more, confirm that the SIM card is inserted correctly and that the antenna is securely fastened. If all is done correctly, you should see the SIM800L board's inbuilt led flashing once every three seconds. As a result, the SIM800L module may create a network using our SIM card. This guarantees the appropriate Operation of our powering circuit. Now that the board is connected, we can build our Python code without an RPi.

Making Calls and Sending SMS on a Pi 4 with a SIM800L Component

You may find the entire Python script for the RPi interactive voice response system at the end of this tutorial. Due to the size of the code, it is not practical to detail every step. It is incorporated with comments and procedures to do reading and comprehending the code easier. Importing the necessary header files allows us to launch the program. We used the serial module to enable the serial connection between the Raspberry Pi and SIM800L components. The pygame software is employed to play audio, in this case, audio files. We also have the time component to cause delays, so that's something. You don't need to install anything new because the Buster OS comes pre-installed with all three packages.

import serial #for serial communication with GSM SIM800L

import time

import pygame #to play music

def SIM800 (Command)

Using this method, an AT command can be sent from a PI computer to a SIM800L device and then received in return. All AT commands transmitted to SIM800L must be written in ASCII and conclude with the character "rn." Before sending them to SIM800L, this method appends the character "rn" to all our AT instructions and transforms them into the ASCII character set. Once the response has been read, it decodes the ASCII data so we may utilize them in our code.

def SIM800(command):

    AT_command = command + "\r\n"

    ser.write(str(AT_command).encode('ascii'))

    time.sleep(1)

    if ser.inWaiting() > 0:

        echo = ser.readline() #waste the echo

        response_byte = ser.readline()

        response_str = response_byte.decode('ascii')

        return (response_str)

    else:

        return ("ERROR")

def wait_for_SIM800()

This function is quite similar to the one before, except instead of sending a value to the SIM800 component, it just awaits a response from the SIM800L module. As a result, it returns the answer when it receives one.

def wait_for_SIM800():

    echo = ser.readline()  # waste the echo

    response_byte = ser.readline()

    response_str = response_byte.decode('ascii')

    return (response_str)

def Init_GSM()

The GSM modem is ready for interactive voice response operations after being initialized. Before sending particular AT instructions to the Gsm modem to activate the message and DTMF receiver modes, it sends an "AT" to check for the module and waits for an "OK" response. In addition, it turns off all alerts so that we won't be distracted throughout a call by texting notifications.

def Init_GSM():

    if "OK" in SIM800("AT"):

        if ("OK" in (SIM800("AT+CLCC=1"))) and ("OK" in (SIM800("AT+DDET=1"))) and ("OK" in (SIM800("AT+CNMI =0,0,0,0,0"))) and ("OK" in (SIM800("AT+CMGF=1"))) and ("OK" in (SIM800("AT+CSMP=17,167,0,0"))):  # enble DTMF / disable notifications

            print("SIM800 Module -> Active and Ready")

    else:

        print("------->ERROR -> SIM800 Module not found")

def play_wav (file_name)

After answering the call, the following method is employed to play the audio files. We have recorded audio files like "cancel.wav," "confirm.wav," and "intro.wav" stored. Following the user's keyboard answer, we must play every one of these files. We can use the play audio function to play any audio file of our choosing. Assume that your Python script is stored within the same folder as these audio files.

def play_wav(file_name):

    pygame.mixer.init(8000)

    pygame.mixer.music.load(file_name)

    pygame.mixer.music.play()

    #while pygame.mixer.music.get_busy() == True:

        #continue

def Call_response_for (phone_number)

The software's most crucial feature is this one. It obtains the mobile number that has to be called and offers the callback for that session. Any response, such as NOT REACHABLE, CALL REJECTED, CONFIRMED, CANCELED, etc., can be given in return. The function calls a specified phone number using AT instructions and then plays recorded audio. The system then listens for the caller's DTMF reply and, depending on that reaction, plays the relevant recorded voice before informing us of the caller's choice. The method will also respond with that information if the caller declines the call or cannot be reached.

def Call_response_for (phone_number):

    AT_call = "ATD" + phone_number + ";"

    response = "NONE"

    time.sleep(1)

    ser.flushInput() #clear serial data in buffer if any

    if ("OK" in (SIM800(AT_call))) and (",2," in (wait_for_SIM800())) and (",3," in (wait_for_SIM800())):

        print("RINGING...->", phone_number)

        call_status = wait_for_SIM800()

        if "1,0,0,0,0" in call_status:

            print("**ANSWERED**")

            ser.flushInput()

            play_wav("intro.wav")

            time.sleep(0.5)

            dtmf_response = "start_over"

            while dtmf_response == "start_over":

                play_wav("press_request.wav")

                time.sleep(1)

                dtmf_response = wait_for_SIM800()

                if "+DTMF: 1" in dtmf_response:

                    play_wav("confirmed.wav")

                    response = "CONFIRMED"

                    hang = SIM800("ATH")

                    break

                if "+DTMF: 2" in dtmf_response:

                    play_wav("canceled.wav")

                    response = "CANCELED"

                    hang = SIM800("ATH")

                    break

                if "+DTMF: 9" in dtmf_response:

                    play_wav("callback_response.wav")

                    response = "REQ_CALLBACK"

                    hang = SIM800("ATH")

                    break

                if "+DTMF: 0" in dtmf_response:

                    dtmf_response = "start_over"

                    continue

                if "+DTMF: " in dtmf_response:

                    play_wav("invalid_input.wav")

                    dtmf_response = "start_over"

                    continue

                else:

                    response = "REJECTED_AFTER_ANSWERING"

                    break

        else:

            #print("REJECTED")

            response = "CALL_REJECTED"

            hang = SIM800("ATH")

            time.sleep(1)

            #ser.flushInput()

    else:

        #print("NOT_REACHABLE")

        response = "NOT_REACHABLE"

        hang = SIM800("ATH")

        time.sleep(1)

        #ser.flushInput()

    ser.flushInput()

    return (response)

def send_message (message, recipient) 

The send message method in this program enables us to send messages in addition to making calls and receiving responses. The message is sent after it receives both the text and the participant's mobile number.

def send_message(message, recipient):

    ser.write(b'AT+CMGS="' + recipient.encode() + b'"\r')

    time.sleep(0.5)

    ser.write(message.encode() + b"\r")

    time.sleep(0.5)

    ser.write(bytes([26]))

    time.sleep(0.5)

    print ("Message sent to customer")

    time.sleep(2)

    ser.flushInput()  # clear serial data in buffer if any

def incoming_call()

The very last method I created for this project allows you to figure out who is calling by their phone number. Some people might attempt to call back to this line because this SIM card will make calls to new persons. If so, you can use this function to see which line is making the call, send them a message, or create a follow-up call if necessary.

def incoming_call():

    while ser.in_waiting: #if I have something in the serial monitor

        print ("%%Wait got something in the buffer")

        ser.flushInput()

        response = SIM800("ATH") #cut the incoming call

        if "+CLCC" in response:

            cus_phone = response[21:31]

            print("%%Incoming Phone call detect from ->", cus_phone)

            return (cus_phone)

        else:

            print("Nope its something else")

            return "0"

    return "0"

It's time to create the main program, in which we will employ all of these functions to accomplish something nice now that all of them have been defined. The customer id and phone number are now being hard-coded for demonstration purposes, but you can obtain them using a Shopify API request or retrieve them from a spreadsheet as needed. For testing purposes, we use the customer's identity, "AISHA," and phone number, "9877XXXXXX."

cus_name = "Aisha"

cus_phone = "968837XXXX"

We will begin a serial conversation with a 15-second delay at 9600 baud speeds within the main endless while loop block. Considering that various SIM800L components may operate at a different baud rate, ensure that you enter the correct COM folder and baud rate in this field.

ser = serial.Serial("/dev/ttyUSB0", baudrate=9600, timeout=15)  # timeout affects call duration and waiting for response currently 30sec

print("Established communication with", ser.name)

After making the call and getting the necessary reply from the consumer, we will send the recipient a text based on their response. For instance, if the answer is validated, we can send a message about it. Likewise, we can alter the message if the client provides a different reply.

response = Call_response_for(cus_phone) #place a call and get response from customer

print ("Response from customer => ", response)

if response == "CONFIRMED":

    text_message = "Hi " + cus_name + ". Your booking has been confirmed.Thank you. -Circuitdigest"

    send_message(text_message, cus_phone)

if response == "CANCELED":  # if the response was to cancel

    text_message = "Hi " + cus_name +. "Sorry that you have decided to cancel your booking. If you canceled by mistake, kindly contact us by phone. -Circuitdigest"

    send_message(text_message, cus_phone)

if ((response == "CALL_REJECTED") or (response == "REJECTED_AFTER_ANSWERING")):  # if the response was rejected

    text_message = "Hi " + cus_name + ". We from circuitdigest.com have been trying to reach you, to confirm your 

    booking. You will receive another call within a few minutes, and we kindly request you answer it. Thank you"

    send_message(text_message, cus_phone)

Testing our interactive voice response system on the Pi

Just double-check the connectors and turn on the GSM component board and RPi. Make that the appropriate COM port is listed in the code before running the application; in my instance, it was "/dev/ttyUSB0." After that, check that the sound is set to atrioventricular ( av ) node jack by right-clicking on the loudspeaker icon and that the audio level is low.

Simply alter the contact number and client name in the following step to your liking, or modify the program to retrieve the information from an excel spreadsheet or cloud Application programming interface, and our automated interactive voice response is ready to go.

The application will dial the specified number and listen for DTMF responses. Additionally, it will play relevant sound clips and transmit a text message dependent on the recipient's response. Below are a few sample messages that I obtained on my cellphone.

The audio tracks and texts can easily be changed to meet your application's needs. I hope the assignment was enjoyable for you and that you learned something.

RASP-IVR application

  1. Call routing within a company is among the most popular uses of an interactive voice response system. Previously, you would employ a switchboard controller or receptionist to answer all callers and direct callers to the appropriate line. When answering customer calls, an interactive voice response system is beneficial. A caller may be given a menu of choices and inquiries about the system's type of call. If possible, the system will respond to the more typical inquiries while referring the less common questions to qualified experts.

  2. Interactive voice response systems are excellent for getting detailed, current info from databases—for instance, movie times. On a significant database, the weekly movie lists are updated. The cinema theater's website can also be filled with information from these databases. When making a call to the cinema, the caller can use the keypad or voice instructions to search the database for movie timings. The same technology can be used to check account balances, evaluate the latest credit card payments, check flight times, refill medicines at a pharmacy, plan auto maintenance, and register for university classes. The list is endless.

  3. Interactive voice response systems are helpful for sales as well. A sales department can create an interactive voice response order form, enabling callers to complete it on their phone's keyboard. When the form is finished, the computer can send a copy to a sales team representative through fax or email. The sales department could also use the IVR as a virtual flyer that highlights the characteristics of a good or service and offers customers the chance to speak with a live agent at any time.

  4. Marketing teams and election pollsters can use the interactive voice response systems' outgoing call functions. A political campaign may send a voicemail message with a phone-in survey for voters to complete. An advertiser may determine whether a buyer is interested in his goods or services. They might press a key to speak with a sales representative if interested in the advertiser's computerized pitch.

  5. Electronic alert systems may also be combined with interactive voice response systems. Let's assume that your company has worldwide workers working from home. Worker contact details, such as home phone numbers, mobile phone numbers, fax numbers, pager numbers, email addresses, etc., can be coded into the interactive voice response system. The IVR prog will use every form of contact until a connection is established if a call has to be directed to that worker.

  6. Transcribing health records is an intriguing application of interactive voice response technology. Currently, doctors record their patient notes and submit the audio to a service that performs medical transcription. However, a doctor may contact the interactive voice response system, record his notes, and have a filed transcript emailed to their office thanks to advanced speech recognition software.

  7. The creators have been collaborating with a few Rwandan health industry partners. The developers want to create solutions that will expand the coverage and reach of the partners' already-existing face-to-face operations. Interactive voice response can extend services to clients who cannot travel to existing facilities. Interestingly, one non-partner frequently visits remote communities and considers the interactive voice response as a tool for drawing people to these outreach efforts. By providing automated processes that operate outside of regular business hours or by gathering and analyzing customer data, the IVR can lessen the employees' workload. One partner will use this function to computerize donor-sponsor reports. In the opinion of one of the partners, the interactive voice response is a small step towards a more significant switch from hard copies to soft copies. Before conversing with a counselor, the web service might pull up the patient's EHR using information gathered by the IVR synchronized with the hard copy to soft copy system and the interactive voice response. After the call, a text message is sent to the customer, and another is sent to the dispatch center that is most convenient for the client.

Advantages of RASP-IVR

Using its speech and texting platforms, the RASP-interactive voice response system opens the door to creating unique apps that can assist with contextually relevant difficulties. Features including automated call rerouting, user-selected content, caller audio content recording, text message-based engagement, data collecting, and tailored resources that can be accessed on caller ID are all made possible by the RASP-Interactive voice response system.

We acknowledge, however, that interactive voice response-based solutions, like all technology, must be developed to enhance current human development efforts. The following are IVR-based systems' benefits, although certain limitations are covered below.

  • A voice-only system is more appealing to people with limited literacy

 than texting or internet-based apps. 

  • To use an IVR system, users just dial a number on their readily available cell phone, as they would with any other service.

  • I am using a mobile phone to get entry to discreet and stigma-free services like psychological counseling, and family planning might be advantageous.

  • Interactive voice response systems can reconnect dropped calls thanks to caller ID and track many simultaneous activities from the same telephone.

  • Finally, the speech signal is informative, providing estimations of the speaker's age, body weight, sexuality, anxiety, and other health indicators.

 

Interactive voice response development Obstacles

Some drawbacks exist, though, which must be weighed against the advantages. Since the interactive voice response system is language-dependent, the prompts must be recorded in users' native tongues. Interactive voice response ern systems are constrained by emerging communities' cultural and social conventions, even though the telephone interface is familiar.

Users more accustomed to interpersonal communication may resist automated data inquiries and exchanges. To that end, one of the partners believes that in-person demonstrations of the IVR system to clients will provide the most positive results.

Another barrier we discovered for people with few resources was the lack of phone access. Due to fluctuating electricity prices and supply, charging a phone is not always possible. In many cases, people will use various telephones, or multiple people will use the same phone. It's possible that follow-up calls and texts won't be received.

In addition, having a shared phone line can raise questions and suspicions about who is calling. Finally, there are several ways in which users' privacy could be compromised by IVR technologies, from the storage and transmission of data to the reporting of such data to the difficulty of preventing one user from accessing the information of another.

The existing system demands developer-side programming expertise that a CBO might not have access to. The dongle's inability to talk to the Raspberry Pi occasionally resulted from a hardware malfunction. The network failed in some cases. When making a mobile-to-mobile call, one internet provider did not forward the touch-tone signals.

Unfortunately, the system can only handle a single call at a time, making it unsuitable for deployments with a high rate of incoming calls. Our co-operators who expect low call volumes can benefit from the RASP-IVR. A shift to a more scalable approach would be justifiable if volume increased.

Conclusion

Through this tutorial, we have gained a thorough understanding of IVRs and their inner workings. We proceeded to assemble our own Raspberry Pi 4 IVR with a handful of components and some test audio samples. We have investigated the mechanism behind this system and spoken about some of the primary uses for the IVR system. We also had a look at some of the advantages and disadvantages of using it. The following tutorial will teach you how to connect a USB barcode scanner to a Raspberry Pi so that you can read 2D barcodes.

Create a WiFi Extender with Raspberry Pi 4

Where To Buy?
No.ComponentsDistributorLink To Buy
1Raspberry Pi 4AmazonBuy Now

Introduction

Welcome to the following tutorial in our raspberry pi four programming tutorial. In the previous tutorial, we learned how to build a raspberry pi four website monitoring tool. We say how beautiful soap is applied in browser automation tasks such as tracking website activities. But in this guide will discover how to utilize our Raspberry Pi 4 as a wifi repeater. If you want a lag-free wireless network, a wifi Wireless Adapter is exactly what you need. You may want to set up a wifi repeater if your signal is weak in certain rooms of your home or if you have entirely lost service in one or more rooms that are too small to accommodate your current network setup. If you want to increase the range of the wifi connection without breaking the bank or wasting a lot of energy, consider getting a Raspberry Wifi Extender. Compared to a wireless access point, a wifi extender has significant differences.

The critical distinction is that instead of using an Ethernet cable, it uses a WiFi adapter to connect to the internet. Two WiFi adapters are required to finish this tutorial, and one of them must be able to function as an access point. You might expect a significant slowdown in your network speed when using the WiFi repeater. That's because devices can't join your network until your initial WiFi communication has gone through and been rebroadcast from your Pi 4.

Our Virtual private network Access Point tutorial, which explains how to set up a VPN connection, complements this one nicely and can be found immediately. It will demonstrate how to install an OpenVPN client and route all network traffic through that app. You'll need two Wireless dongles to finish this guide, at least one of which should be able to function as an AP. Those components are listed below.

Components

  • Raspberry Pi

  • Power Supply

  • WiFi x 2

How WiFi boosters improve your internet connection?

A standard wireless range extender may pick up the signal from your existing wireless router, boost its signal strength using several channels, and broadcast it over a larger region in your house or workplace. Say, for instance, you have your WiFi router put on the ground floor of your building, and you are studying on the 2nd floor of your room. You're trying to link up your computer or mobile device, but obstacles keep popping up. A WiFi Extender, which, once activated, enables wireless connectivity to the router, is an ideal solution for resolving this issue.

How to Setup a WiFi Range Extender

Because it doubles as a Domain Name System (DNS) and a Dynamic Host Configuration Protocol server, the dnsmasq package is essential to setting up our RPi Wireless Extender and doing the bulk of the legwork for this guide. Another program we'll need to use is hosted, enabling us to use one of your wifi gadgets as an AP. Don't forget that you'll need access to a working wireless router and an Ethernet device to bridge your wireless connection; follow this guide. For starters, we'll update the rPi by typing the two commands below into the terminal.

sudo apt update

sudo apt upgrade

Installing dnsmasq and hostapd, two necessary packages, is now a simple matter of running the appropriate commands.

sudo apt install dnsmasq

sudo apt install hostapd

It's important to prepare the wlan0 network connection before moving forward. It is possible to skip to Step 5 if you have previously established a wireless network connection. Alternatively, you can access the WPA-supplicant.conf script by issuing the command below on your RPi:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Incorporate the following into this file, switching out "ssid" for the network's id and "psk" for the passcode.

network={

        ssid="networkname"

        psk="networkpassword"

}

Save the script by clicking Control + X, followed by Y, then finally, ENTER key once you have provided your network username and password. Assuming you successfully installed all the packages and connected to the wifi network, the next step would be to configure dhcpcd to provide our RPi with a permanent IP address. This can be accomplished by adding the command below to the dhcpcd.conf script:

sudo nano /etc/dhcpcd.conf

The lines below, added to the file's end, will configure the wlan1 connection as desired. Please insert the lines below.

interface wlan1

static ip_address=192.168.220.1/24

static routers=192.168.220.0

Clicking Control + X, followed by Y, and then Return will save the file and allow us to exit the program. After making changes to the DHCP configuration, it is necessary to restart the service.

sudo service dhcpcd restart

Next, we'll change the configuration file for hostapd by entering the following command into the terminal of your RPi.

sudo nano /etc/hostapd/hostapd.conf

The following lines, which must be written into this file, establish the framework for our communication with the WLAN hardware. The only two lines in this file that warrant serious consideration are the ssid= and WPA passphrase= entries. Some wifi devices have particular driver needs; thus, you may need to modify the driver= line to point to the most significant driver for your device. A quick Google search will do the trick if you need help figuring out which driver to use.

interface=wlan1

driver=nl80211

hw_mode=g

channel=6

ieee80211n=1

wmm_enabled=1

ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40]

macaddr_acl=0

ignore_broadcast_ssid=0

auth_algs=1

wpa=2

wpa_key_mgmt=WPA-PSK

rsn_pairwise=CCMP

ssid=Pi3-Extender

wpa_passphrase=raspberry

To restrict access to your wifi booster from prying users, you need to modify your WPA passphrase to a strong password of your choosing. We can save the file and exit the program using Ctrl + X, Y, and Enter. Once that is complete, we can make changes to the hostapd configuration script in /etc/default/ and /etc/init.d/. It must first read these files for hosting to use the newly produced configuration file. The command below will open the initial of these two script for editing.

sudo nano /etc/default/hostapd

The following line needs to be updated in this document.

Find:

Instead, substitute

DAEMON_CONF="/etc/hostapd/hostapd.conf"

Clicking Control + X, followed by Y, and ENTER will save the file and allow us to exit the program. We'll modify the next script we need to set up now, which can be located in the init.d folder. The command prompt will allow us to make changes to the file:

sudo nano /etc/init.d/hostapd

The lines below need to be updated in this document.

Find:

Change to:

Tapping Control + X, followed by Y, and afterwards, ENTER will save the file and allow us to exit the program. Now that hostapd is ready to go, we can move on to configuring disease, but first, we need to relocate the default config file. Let's use the command below to accomplish this:

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.Orig

After relocating the old configuration script, you can start working on your new file. The command below will be used to make the new file and make any necessary changes:

sudo nano /etc/dnsmasq.conf

Including the lines below this file is recommended, instructing the dnsmasq service about handling incoming connections.

interface=wlan1       # Use interface wlan1  

listen-address=192.168.220.1   # Specify the address to listen on  

bind-interfaces      # Bind to the interface

server=8.8.8.8       # Use Google DNS  

domain-needed        # Don't forward short names  

bogus-private           # Drop the non-routed address spaces.  

DHCP-range=192.168.220.50,192.168.220.150,12h # IP range and lease time  

Pressing CTRL + X, followed by Y, and afterwards ENTER will save the file and allow us to exit the program. Now we must set up our Raspberry Pi to relay all wlan1 traffic over to wlan0. To begin, modify the sysctl.conf config file using the command below to activate it:

sudo nano /etc/sysctl.conf

locate the line below in this file and delete the hash symbol (#).

Locate:

substitute with

net.ipv4.ip_forward=1

Pressing Control + X, followed by Y, and then ENTER will save the file and allow us to exit the program.

In any case, if you don't want to wait for it to activate on the next boot, you can use the command below to make it happen instantly:

sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

Now that IPv4 Forwarding is working, we can set up NAT across our wlan0 and wlan1 networks. Essentially, this will redirect all wireless network traffic from our AP to our ethernet link. To implement our new iptable rules, we must execute the following commands:

sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE  

sudo iptables -A FORWARD -i wlan0 -o wlan1 -m state --state RELATED,ESTABLISHED -j ACCEPT  

sudo iptables -A FORWARD -i wlan1 -o wlan0 -j ACCEPT

If you have any problems after typing the above lines, try doing a sudo reboot on the Pi. The new rules we create will need to be saved somewhere so that they can be reloaded into iptables after each reboot of the Raspberry Pi. Use this command to store our modified set of rules permanently:

sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"

Now that we have our new rules written down, we need to ensure they have loaded back in every time the system restarts. Changing the RC.local file is the simplest solution. To begin making changes to the file, type in the following command:

sudo nano /etc/rc.local

Since we're already here, we might add this line that follows the file. See that this line is visible from Exit 0 and higher. This line extracts our iptables configuration settings. Iptables takes the rules from the ipv4.nat file and applies them.

Locate:

Add this:

iptables-restore < /etc/iptables.ipv4.nat

We can now save the file and exit the program. We may conclude by using systemctl to enable the two services we just started. Put these two commands to work:

sudo service hostapd start

sudo service dnsmasq start

If everything went smoothly up to this point, you should now have a fully functional Pi Wireless AP, which you can test by linking any of your wifi devices to it with the Id and WPA Passcode you established earlier the tutorial. A restart may be all that's needed to get things back up and running when an error has been detected. This will guarantee that the Raspberry Pi's re-enabled features work correctly after restarting. To force a reboot of the Raspberry Pi, type the following command:

sudo reboot

Implementing a Virtual Private Network on the Pi 4 wifi Extender

Any device, regardless of whether it has VPN software installed, can be routed through a secure connection by connecting to a VPN Access Point since all traffic from that device will immediately begin passing through a VPN tunnel after being connected to your Pi 4 wireless access point. This article expands upon our earlier guide on setting up a wifi access point by making one significant adjustment: redirecting all data traffic through a virtual private network. Using either VyprVPN or NordVPN, we'll demonstrate how to set up this secure tunnelling protocol. For this guide, we tested several different VPN services, which proved to be among the most dependable in terms of speed and privacy policies.

Installation of OpenVPN on the App

This guide will explain how to use your VPN provider's software to establish an encrypted tunnel between your local network and the wider internet. This guide will discuss using VyprVPN and NordVPN as Virtual Private Networks. Both of these met our requirements for dependability, speed, and sound privacy protections, so we settled on them. OpenVPN will be used for the VPN connection; as both services, we'll discuss offering complete support for OpenVPN standard and supply us with the necessary.open files to get it up and running. Now that everything has been up to date, we can set up OpenVPN, the program we'll be using. Using the terminal, we may accomplish this by inputting the command below:

sudo apt install OpenVPN -y

Let's now move into the OpenVPN directory, where we'll keep all the files and configuration settings for our Raspberry Pi VPN server. Let's use the change folder command to go directly to that folder.

cd /etc/openvpn

We need to generate an authorized file now that we are in the OpenVPN directory. The auth.txt file we will make is where your VPN login information will be stored.

Create the file initially by issuing the below nano instruction.

sudo nano /etc/openvpn/auth.txt

Copy and paste the example login and password below into the file. We will configure OpenVPN to use this file as its login credentials whenever you connect to your VPN.

When you're finished making changes, save the script, and we'll go on.

Where to Locate the VyprVPN OVPN script

First, execute this command to verify that you are still in that OpenVPN folder before proceeding with downloading the VyprVPN installation files. Of course, you'll need a VyprVPN account before using these files.

cd /etc/openvpn

Once the VyprVPN OVPN files have been downloaded, you can access the configuration directory. We will be loading these components to connect to Vypr's server network. Download the archive from VyprVPN's servers using the wget command.

sudo wget -O vyprvpn.zip \

https://support.goldenfrog.com/hc/article_attachments/360008728172/GF_OpenVPN_10142016.zip

After transferring the zipped file to the RPi, we must extract it to use its contents. The unzip command below can be executed in the terminal to extract the file.

sudo unzip vyprvpn.zip

Now that the.ovpn files have been generated, you can explore them in the corresponding directory. Make a note of the target server's location, as you'll need to supply it in a later operation.

cd OpenVPN256

Execute the command below to display a list of all.ovpn files are currently stored here. Remember the filename of the desired location; for example, "USA - San Francisco.ovpn" would be remembered if a connection to San Francisco, California, in the U.s was desired.

ls

The required ovpn file should be transferred from the OpenVPN256 folder to the OpenVPN folder. Since we'll be making modifications to the file, having it under the OpenVPN directory will make autostarting the daemon much less hassle. Our sample file may be relocated to the correct folder with a command like this. You must be in the VyprVPN folder to run this command. You'll need to rename the extension if you want OpenVPN to recognize and use the file.ovpn to.conf. We will modify the file after moving it to make it more user-friendly.

sudo mv "USA - San Francisco.ovpn" "../USASanFran.conf"

After that, you can proceed to "Setting up the Virtual private network access point," where you'll find detailed instructions on modifying the.ovpn file and setting the VPN to launch when the system boots automatically.

Accessing NordVPN's OpenVPN configuration files

After finishing the auth.txt file, head to NordVPN and download the necessary.ovpn configuration files. Please note that you will need a NordVPN subscription to use these files. However, before we proceed, let's double-check that we're already in the correct directory by entering the following command into the terminal.

cd /etc/openvpn

Now, navigate to the ovpn area of the NordVPN webpage to get the OpenVPN configuration files. Locate the server you want to use on this site, right-click on "Download UDP," and select "Copy Address" to copy the link. Simply replace the URL in the following command with the copied URL. We'll be using the au514 server in this example.

sudo wget https://downloads.nordcdn.com/configs/files/ovpn_legacy/servers/au514.nordvpn.com.udp1194.ovpn

The downloadable file should now be renamed. You can save some time in the tutorial by giving the file a new name before using it. Additionally, you must convert the file extension from.ovpn to.conf. This modification is required for OpenVPN to recognize the file and incorporate it into its startup process when autostart is enabled. See how easy it is to rename a file with the mv command by looking at our working example. To make the au514.nordvpn.com.udp1194.ovpn filename more manageable while still providing sufficient information about which server it is for, we reduced it to just au514.conf.

sudo mv au514.nordvpn.com.udp1194.ovpn au514.conf

The section "Setting up the Virtual private network access point" can be accessed once the file has been renamed. In this part, you will learn how to connect to an OpenVPN server and redirect traffic through it.

Configuring the Virtual Private Network Access Point

Since we have a working.ovpn file, we need to tweak it so that our RPi Virtual private network access point can connect to the internet. Start by executing the following command, changing the filename to the one you want to use.

sudo nano au514.conf

The following line in this file has to be edited, so go ahead and do a search. To quickly locate this passage, press CTRL + W. This modification will cause the Vpn client to load the auth.txt file whenever it is opened automatically.

Find

Switch to

After making changes, use CTRL + X, Y, and enter to save the script.

After that is complete, we can verify that OpenVPN is connecting to our VPN service, NordVPN. When we run the command below, it will overwrite the existing one.conf (.ovpn) script with the one we edited in the previous section. We'll use the "au514.conf" configuration file as an illustration.

sudo openvpn --config "/etc/openvpn/au514.conf"

While this operation establishes the connection, it will generate a large amount of text; the most important is the format we've demonstrated below. If you get similar results, you are successfully connected to your OpenVPN provider's servers.

After a successful trial run of the OpenVPN client, we may terminate it and move on to making it launch automatically at system boot and adjusting firewall routing to force all traffic to go through the Virtual private network tunnel. Hit Control + C to cancel the action you're now performing. Now that we have a functional VPN, we can use iptables to redirect the wlan0 traffic through the VPN instead of directly to an Ethernet-like we did in the previous wireless AP tutorial. But before we do so, we need to clear away our existing iptables, which we can accomplish by using the three lines below on the RPi.

sudo iptables -F

sudo iptables -t nat -F

sudo iptables -X

Finally, we can install the updated iptables by issuing the command below, which will direct all wlan0 traffic through the tunnels.

sudo iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

At this point, we may save our iptables and remove the rules we previously set up for our Wireless AP guide. To accomplish this, enter the command below into the RPi terminal.

sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"

Since the firewall is operational, the Vpn client's config file will need to be modified. To start modifying the file, type in the following line.

sudo nano /etc/default/openvpn

The following line should be uncommented, and "all" should be substituted with a file name you edited in the previous Steps when you return to this file. Please see the example we've provided below.

Locate

Instead, substitute

autostart="au514"

After making the necessary edits, press Control + X, followed by Y, and finally, return the key to save and quit the Vpn config file. Rebooting your RPi should now allow you to test out your Setup. At boot-up, it must connect to the VPN, create the tunnel, and route all wireless signals.

sudo reboot

When using a VPN, the best way to ensure it is functioning correctly is to compare your Internet address to that of a device that is not linked through a Private network. On the other hand, both NordVPN and VyprVPN, two of our examples, clearly indicate whether you are "Protected" or not at the very top of their websites. If you want the same results as those shown above, you'll need to make one tiny adjustment and use the below commands. Most noticeably, we will now use our wlan1 connections to tunnel traffic rather than wlan0.

sudo iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

sudo iptables -A FORWARD -i tun0 -o wlan1 -m state --state RELATED,ESTABLISHED -j ACCEPT

sudo iptables -A FORWARD -i wlan1 -o tun0 -j ACCEPT

Now your raspberry pi should be set up as a wifi extender

Pi 4's strengths as a wifi booster

Depending on several factors, you can take advantage of a Wireless extender in various settings. These wifi Extender Benefits analyze why you need one and the amount of money you may save using one.

  • Increased Protection in Open Space

When your home has a lot of electronic devices, a wifi Booster or repeater is a robust solution for covering wireless communication coverage. Connecting a wifi extender to your home's network can increase the signal's reach, which is one of the device's primary benefits. It will boost your router's password by acting as a repeater.

  • The Wireless Extender is Simple to Install

Plug-and-play capabilities and automated configuration are standard on modern wifi range extenders. It's user-friendly enough that you won't need to study a technical manual before putting it to work. You can take a product straight from the box, put it into an electrical outlet, and start using it. The Setup and maintenance of a wifi range extender are free of charge.

  • The existing Router Will Be Reused.

Since the Wireless repeater and the original router share the same Id, you can continue to use the original router with the repeater. Range extenders can be used to convert an existing router into a repeater, saving you the trouble of having to dispose of the device. You can reuse the signal in different rooms without running additional cables by setting up a wifi range extender. You can get the most out of your outdated network gear.

  • Budget-Friendly

Adding a wifi repeater to your network will increase its coverage without requiring additional hardware. One of the benefits of wireless extenders is the money they can save you. Moreover, a Wireless extender setup requires no extra wires or labour, making it the simplest to set up. These gadgets are completely pre-configured, so there is no need to spend money on laborious Setup or special cables.

  • Connectivity for a large number of users

A key drawback of having a shared wifi network is the loss of connection speed every time a new user joins your wireless network. Imagine you're attempting to load Wikipedia while someone else is streaming in high volume. It's common to run into this issue when the number of users on the network rises, and the available bandwidth decreases. A wifi range extender is helpful for these people because it allows them to connect to the network from anywhere in the house by using various channels and frequencies.

Limitations of Wireless-Fi Extender Technology

While a wifi repeater is a fast, innovative, and cheap solution to wifi coverage concerns, it is essential to note that it is not a magic bullet. You may encounter these drawbacks of wifi Extender, which may cause you to reevaluate your position.

  • Bad Reception

If your wifi adapter is emitting a weak signal, a Wireless extender can help. If your router is older or unsecure, you may run into any of these primary issues as a consequence of utilizing a Wireless Booster at home. Repeaters typically reduce network bandwidth for connected PCs by two, so upgrading your modem and wifi extender is not a good idea. A signal is received, processed, and then repeated on many channels by a repeater. Both the PC and the router take part in this procedure. Because of this cause, the signal intensity is weakened.

  • Low Speeds

There may likely be a decrease in Wireless internet speed because of the signal's passage through the repeater due to signal weakness. As the wifi range extender strives to boost the movement within your residence, the quality of your network will inevitably decline.

  • Issues of Incompatibility

However, setting up a repeater from a different manufacturer to work with the router can become a chore. There could be compatibility or operability concerns as a result of this. Generally, it's best to use parts and software made by the router's original manufacturer.

In most cases, the repeater will have a unique password and SSID from the router. Launching a new brand name comes with its share of challenges.

  • Adherence to emerging wifi standards

It is wise to verify how often a wifi range extender's software is updated before purchasing. You should think about this if you want to use it for a long time. Wireless range extenders can become unusable if their firmware is out of the current. Don't buy anything if the fixes aren't readily available as patches.

Conclusion

This tutorial taught us how to build a raspberry pi four wifi extender to increase the wifi range of our home wifi network connection. We also looked at how we can configure a VPN for the same pi four wifi extender to offer security to our home network. Now we can try out more advanced ideas like how to boost the signal strength of the wifi extender without interfering with the internet service provider. Using a Raspberry Pi and a SIM800L in the following tutorial, we'll set up an IVR system that can dial numbers and send text messages automatically.

Monitor a Website with Raspberry Pi 4 using Python

Where To Buy?
No.ComponentsDistributorLink To Buy
1Raspberry Pi 4AmazonBuy Now

Introduction

Welcome to the next tutorial in our raspberry pi programming. In the previous tutorial, we learned how we could run Xbox cloud on our pi 4. We found that we can run any game easily without lag or having raspberry pi shut down, proving that pi 4 is quite a powerful minicomputer. However, this tutorial will demonstrate how to use Python on raspberry to monitor websites. This Python program will execute on Pi 4 and watch over a website, alerting you whenever it changes or goes down. This is accomplished by keeping a straightforward webpage duplicate locally and monitoring for updates. This webpage monitor is straightforward, so it should work well on pi Zero. Throughout this tutorial, we will demonstrate how to develop your custom code to monitor a webpage. With this information, you ought to be able to modify the script to meet your unique requirements.   However, for email updates to function, a few settings must be made in the core code.

The lack of an interface makes this project ideal for a headless Pi. Even though this tutorial concentrates on Pi 4, the core code can be used on any device that supports Python 3. That implies that, if you'd want, you could even execute this program on a Windows computer.

Equipment

  • Raspberry Pi

  • Micro SD Card

  • Power Supply

  • Ethernet Cable or Wi-Fi

How to get your Raspberry Pi ready for the webpage monitor

Ensure you have all the components you need to execute your website monitoring program before you begin. These actions include installing Python 3 and the necessary Python libraries. Updates to the package registry and any current packages come first. To make these upgrades, we must execute the following lines in the console on our Pi 4.

sudo apt update

sudo apt upgrade -y

We must make sure that Py 3 and its application manager, "pip," are both installed on the mini-computer. To guarantee that the two packages are loaded, use the below command.

sudo apt install python3 python3-pip

Installing the necessary Python libraries is the last step before writing the program to monitor our web pages. Download the packages for queries, beautiful soup4, and lxml with this line.

pip3 install requests beautifulsoup4 lxml

Coding a website monitor for your Pi 4

Having completed the installation of the necessary packages, we can begin drafting our elementary website monitoring code. So that you can understand how everything functions, we'll break this down into its parts. Though the nano editor will work, a more capable integrated development environment (IDE) like Visual Studio Code is recommended. Get started on the Python code to keep an eye on a website with your Raspberry Pi. Websitemonitor.py is the name of the script we'll be working with throughout this guide. Launch nano and enter the following command to begin writing the script.

nano websitemonitor.py

Writing a Simple Website Monitor in Python

Before we get too far into coding a complex website monitoring program for our RPi, let's start with the simplest possible solution. We shall craft a basic script to retrieve a web page, check its contents against a known master copy, and output an appropriate message if any changes have been made. At the beginning of any script, we must import the packages we'll be working with. These scripts' launch requires the "os," "sys," and "requests" packages.

  • Os: To communicate with the os, you will need to install the package known as os. For our purposes, we'll be using this to store a copy of our most recent website request in a cache. To detect any modifications, we will check this cache.

  • Sys – For any arguments supplied to the program, we utilize the sys library to retrieve them. We'll allow the user to provide the URL of the website and the name of the cache in this case.

  • Requests – Python can only send requests thanks to the requests package.

This allows us to take a certain website's content and save them.

import os

import sys

import requests

Coding the has_website_changed() method

Most of our logic will be handled by a function we'll write next. Two parameters must be set for this function: "has the website changed." The first input, the website URL, contains the website's address. We will make we obtain a request from this point forward. The website name, the second parameter, contains the website's name. This is a condensed form of the name used for the cached file. This method will have 3 potential return values: -1 if the webpage is "not ok," 0 if the webpage hasn't changed, and One of the webpages is altered.

def has_website_changed(website_url, website_name):

Keep in mind that indentation is crucial when using Python. Please make careful to keep the indentation when you complete this function.

Using Python Request Headers

Now that our function is specified, we can move on to implementing it. To initiate a website request, we must first define the headers sent by the request module. We are doing two things with these headers. For starters, there's the "User-Agent" header. Set it to whatever works best for you. For the most part, we are trying to keep things straightforward here.

Second, we instruct the client and the destination server not to store this request in their cache by setting the "Cache-Control" header to "no-cache." There is no guarantee that the requested web server will comply.

headers = {

        'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; PIWEBMON)',

        'Cache-Control': 'no-cache'

    }

Putting in a Request to the Internet Service Provider

The queries package is secure enough to retrieve the passed-in URL with the requested headers. This is a vital part of our RPi's website monitoring software, as it returns the site's current status. This is an example of the get function included in the "requests" package. The "headers" and "website URL" variables are passed in. The result of this query will be saved in the "response" parameter.

response = requests.get(website_url, headers=headers)

Check That We've Gotten a Good Response

Once we've retrieved the website's response, we need to double-check that it was an "OK" response. Simply put, we need to ensure the error code is not below 200 or above 299, inclusive.

If the value is outside our allowed range, we will return "-1" to indicate an error on our end.

if (response.status_code < 200 or response.status_code > 299):

        return -1

Record the Message and Cache Name in a Temp Directory

Let's set up two more variables after verifying the response to ensure we get a proper status code. For now, we'll keep the text of the response we received in a variable called "response text." This variable will allow us to tailor the response wording before saving it. Our RPi website monitor will continue to work with the present form of the answer text for the time being.

As a second step, we set up a variable named "cache filename" to hold the location of our cache's data file. This name will be fabricated using the "website name" argument and the suffix "_cache.txt." If our website's name were to be entered as "theengineeringproject," the resulting filename would be "theengineeringproject cache.txt."

response_text = response.text

       cache_filename = website_name + "_cache.txt"

Making a Cache for a Brand-New Website

It's possible that the current website's URL won't have a cache file when you initially run the program. We use the "path.exists()" function from the "os" package to see if the cache script already exists. If the cache file doesn't already exist, we make one by opening it with the "w" switch. We then append the text of the current response to the file, allowing our Raspberry Pi to check for updates to the website. Given that this is a brand-new request, we will respond with a value of 0 to indicate that the reply has not changed.

If not os.path.exists(cache_filename):

        file_handle = open(cache_filename, "w")

        file_handle.write(response_text)

        file_handle.close()

        return 0

Review the Pre-Cached Reply to a Prior Request

If the program reaches this stage, the "previous response text" variable will need to be updated with the contents of the cache file. Now we'll utilize "r+" with the open function. The following command informs open that we want to be able to read and write to our cache file. After Python finishes reading a file, the stream position is reset to the beginning; thus, we need to use the "seek()" function to return to the beginning. If this is a fresh reply, it will be simpler to truncate the file.

file_handle = open(cache_filename, "r+")

    previous_response_text = file_handle.read()

    file_handle.seek(0)

Verify the Message Text Identical to the Stored Message

We may now check for a match between the answer text and the original one.

If the responses are identical in content, we exit the process by closing the file handle and returning 0. Recall that a value of 0 indicates nothing has changed in the responses. This, together with the else statement, completes our RPi's "has website changed" monitoring function.

if response_text == previous_response_text:

        file_handle.close()

        return 0

How to make a New Cache Whenever the Response Has Changed

If the responses don't match, our RPi has picked up on a change while keeping tabs on the webpage. If the race were being run right now, it would be in position 0. We then append the updated reply to the file once it has been truncated. The file handle can be closed as soon as the writing is done because it is no longer required. Since the answer has shifted, we send back a 1 to show that it has been updated.

else:

        file_handle.truncate()

        file_handle.write(response_text)

        file_handle.close()

        return 1

Formulating the main() method

To continue our RPi-based website monitor, we must now create its main method. This method will be invoked anytime the script is executed. In this script section, we'll mostly be concerned with invoking the function we just made, so it should be rather simple. First, we'll identify the primary role. This is where the real magic happens for our RPi-based webpage monitoring.

def main():

Inspecting the Website for Updates

At this point, we can access the "has website changed()" method we've created. This method's initial and next parameters will be passed in via the "sys" package. The web address (Uniform resource locator) will be the first argument. As for the second, it will become the cache file's title. The function's output is saved in the "website status" variable we created.

website_status = has_website_changed(sys.argv[1], sys.argv[2])

Website Response Printing

Our variables now contain the current online status, so we can use that to inform our output. Our basic website monitoring software for the Raspberry Pi is now complete. We can expand upon this feature to make it possible to communicate via email or text message. This is a basic if, Elif clause that prints out various text based on the input.

if website_status == -1:

        print("Non 2XX response while fetching")

    elif website_status == 0:

        print("Website is the same")

    elif website_status == 1:

        print("Website has changed")

Last, we can wrap up our script by including the call that activates the script's main method. Using an if clause, we can be sure that the program was actually called and not just loaded as a Python package.

if __name__ == "__main__":

    main()

The Final Structure of the basic program

The program can be saved and tested at this stage. What we have provided here is how the completed core code should appear. To save your work in the nano editor, use control + X, followed by Y, and finally, the Return key.

import os

import sys

import requests


def has_website_changed(website_url, website_name):

    headers = {

        'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; PIWEBMON)',

        'Cache-Control': 'no-cache'

    }


    response = requests.get(website_url, headers=headers)

    if (response.status_code < 200 or response.status_code > 299):

        return -1

    response_text = response.text

    cache_filename = website_name + "_cache.txt"

    If not os.path.exists(cache_filename):

        file_handle = open(cache_filename, "w")

        file_handle.write(response_text)

        file_handle.close()

        return 0

    file_handle = open(cache_filename, "r+")

    previous_response_text = file_handle.read()

    file_handle.seek(0)


    if response_text == previous_response_text:

        file_handle.close()


        return 0

    else:

        file_handle.truncate()

        file_handle.write(response_text)

        file_handle.close()

        

        return 1


def main():

    website_status = has_website_changed(sys.argv[1], sys.argv[2])


    if website_status == -1:

        print("Non 2XX response while fetching")

    elif website_status == 0:

        print("Website is the same")

    elif website_status == 1:

        print("Website has changed")

if __name__ == "__main__":

    main()

Let us put the RPi's Web Server Monitoring Basics to the test.

Seeing that our web monitoring software is complete, it can be deployed on the Raspberry Pi. The following procedures will help us test and verify that the script is functioning properly. Our web monitoring script requires elevated permissions before proceeding. By executing the following line on the RPi, we may grant this access to the website monitoring program.

chmod +x websitemonitor.py

Now that the script has the correct permissions, we can run it. When running this script, you will need to know the URL you want to monitor and the name you want to use for its cache. For our example, we will use "https://www.theengineeringprojects.com/” as our URL to monitor and “project” as our cache name.

python3 websitemonitor.py https://www.theengineeringprojects.com/ project

You may check if the website monitor produced the cache file with the ls command in the console. A text document with the extension "_cache.txt" should be present. Take the "theengineeringprojects cache.txt" file as an illustration. If you execute this script, you may find an issue right away. Metadata can be dynamically set on some websites, meaning that even if the content remains unchanged, each request to the site could result in a slightly different response. If you use our website, you'll see that our script will flag the page as modified whenever we update the core code. In the next part, we'll demonstrate how to use Python's lovely soup to tidy up the outcome and remove anything that could falsely flag the webpage as modified.

Utilizing beautifulsoup to enhance the RPi Web Monitor

Here we'll use beautiful soup to enhance Raspberry Pi's web server monitoring. The beautifulsoup Python module is a potent tool for modifying HTML documents. We may eliminate clutter like style and script tags, for instance. You'll need to adjust the script we built in the last step for this one. A new import is added to the script's initialization. This will bring in the bs4 library and its BeautifulSoup module.

from bs4 import BeautifulSoup

Developing a Brand-New cleanup HTML() method method

Our new mission must now begin. Any HTML obtained from the queries package will be cleaned up using this method. As a result, our RPi's behavior during web monitoring will become more standardized. Including the following code in the file will define the new method. The HTML material to be processed will be supplied as the sole input to this function.

def cleanup_html(HTML):

Produce an instance of the BeautifulSoup object.

A fresh instance of BeautifulSoup is instantiated now. The first argument is the HTML string to be cleaned up. We specify the HTML parser we need to utilize in the second argument. XML is used since it is quick and has all the required features. A faster and more efficient program is always welcome in a resource-constrained environment like our RPi, in which we are executing a web monitor.

soup = BeautifulSoup(HTML, features="lxml")

Using BeautifulSoup to clean up the HTML.

With BeautifulSoup, we can parse HTML and get rid of unwanted tags. We strip off the "script," "style," and "meta" tags with for loops as well as BeautfulSoup's "select" method. You'll see that the ".extract()" method is called on each loop iteration. When an element matching the given criteria is located, this function deletes it.

for s in soup.select('script'):

        s.extract()

    for s in soup.select('style'):

        s.extract()

    for s in soup.select('meta'):

        s.extract()

The BeautifulSoup Object returned as a string.

After BeautifulSoup has processed the HTML retrieved by our website monitoring software, we can return it. It's not enough to return your soup object to its current state. Alternatively, the "str()" function must be used to transform it into a regular string.

return str(soup)

Streamlining the Obtained HTML Response

Our "cleanup html()" function is complete; now, we only need to update some other code to use it. Locate the piece of code below and change it. Instead of storing the reply text without thinking, we first pass it through the new method. The "has website changed()" method is where you'll want to put it.

Substitute the following for the sentence.

response_text = cleanup_html(response.text)

How to save the new extended web monitor

Once you've finished editing the script, the program should resemble what we've displayed below. To save your work when using nano, use control + X, followed by Y, and finally, the Return key.

import os

import sys

import requests

from bs4 import BeautifulSoup

def cleanup_html(HTML):

    soup = BeautifulSoup(HTML, features="lxml")

    for s in soup.select('script'):

        s.extract()

    for s in soup.select('style'):

        s.extract()

    for s in soup.select('meta'):

        s.extract()

    return str(soup)

def has_website_changed(website_url, website_name):

    headers = {

        'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; PIWEBMON)',

        'Cache-Control': 'no-cache'

    }

    response = requests.get(website_url, headers=headers)

    if (response.status_code < 200 or response.status_code > 299):

        return -1

    response_text = cleanup_html(response.text)

    cache_filename = website_name + "_cache.txt"

    If not os.path.exists(cache_filename):

        file_handle = open(cache_filename, "w")

        file_handle.write(response_text)

        file_handle.close()

        return 0

    file_handle = open(cache_filename, "r+")

    previous_response_text = file_handle.read()

    file_handle.seek(0)

    if response_text == previous_response_text:

        file_handle.close()

        return 0

    else:

        file_handle.truncate()

        file_handle.write(response_text)

        file_handle.close()

        return 1

def main():

    website_status = has_website_changed(sys.argv[1], sys.argv[2])

    if website_status == -1:

        print("Non 2XX response while fetching")

    elif website_status == 0:

        print("Website is the same")

    elif website_status == 1:

        print("Website has changed")

if __name__ == "__main__":

    main()

The script is ready for further testing. More stable outcomes are expected this time around. We can decrease the likelihood of a false detection by eliminating the "script," "style," and "meta" tags. For instance, every query on our website must not be recorded as "modified" if you follow this pattern.

Implementing email support to the pi 4 web  monitor

Our Raspberry Pi monitoring system is somewhat useless without a way to be alerted when a website has been updated. We'll add to our functionality here by notifying you through email anytime the script notices a shift. Remember that this will necessitate you to have the SMTP information for your email provider at hand. Here, we'll use the SMTP settings for Gmail as an example.

Introducing a Fresh Import

We must import a separate package to make SMTP connections from within Python. Thankfully, Python already has this module built in. Include the following line in your script's imports. This line will make it possible to establish SMTP connections quickly by importing the necessary "smtplib" package.

import smtplib

The Process of Defining Constants to Save Email Information

At the outset of the Python code, we must define certain constants. Below the existing "import" command, add the lines below. The fact that Python doesn't truly support constants doesn't stop us from naming these parameters with all capital letters, though. You shouldn't modify these values while the program is running.

SMTP_USER

Identity for your outgoing mail server (Simple mail transfer protocol) connection, as defined by this constant. This is the account you'll use to access Gmail.

We refer to the value saved in this constant for the simple mail transfer protocol connection.

SMTP_USER='example@gmail.com'

SMTP_PASSWORD

The SMTP user's password must be set inside this constant. This is the password for your Gmail account. Second-factor authentication (which you should enable) requires a separate app passcode.

SMTP_PASSWORD='PASSWORD'

SMTP_HOST

The address or URL to which the SMTP connection should be established is saved in the "SMTP HOST" constant. Here, we'll use the SMTP server information for Gmail as an example.

SMTP_HOST='smtp.gmail.com'

SMTP_PORT

Using this setting, we tell our RPi web monitor which port to use when it detects a change and needs to notify us through email. Using Gmail's port for implicit SSL, as shown below: (port 465).

SMTP_PORT=465

SMTP_SSL

These days, most email providers offer a secure connection (SSL or TLS). Our software will only be able to utilize HTTPS protocol. Ensure the constant is set to True to activate this support. By setting this to False, SSL will be turned off.

SMTP_SSL=True

SMTP_FROM_EMAIL

At long last, we have the option to designate the sender's email address for this message. It must be an account you have created to use this email. If you're using an Email account, this should be a verified email address. You need to set up that email account and domain in your transaction email provider, such as Mailgun.

SMTP_FROM_EMAIL='example@gmail.com'

SMTP_TO_EMAIL

The final required setting is the destination email address for the script to use to send the email. Enter the email address at which you would like to be notified of updates to the website.

SMTP_TO_EMAIL='sendto@gmail.com'

Creating the Email Notification method

Our "email notification()" method can be written when we have declared all necessary constants. This function will take care of the process of connecting to an SMTP server and sending an email. Let's begin by identifying our new role. There will be two inputs for it. 

def email_notification(subject, message):

Forming an SMTP Link

Initiating an SMTP session is the first step of this method. Since we need to accommodate both SSL and non-encrypted connections, we'll have to make two distinct calls separated by an if clause. An SMTP session is established and saved in the "SMTP server" parameter if the "SMTP SSL" flag is set to True. When SSL is not enabled, we also take this action. We include the SMTP HOST and SMTP PORT environment variables when establishing the link.

if (SMTP_SSL):

        smtp_server = smtplib.SMTP_SSL(SMTP_HOST, SMTP_PORT)

    else:

        smtp_server = smtplib.SMTP(SMTP_HOST, SMTP_PORT)

The SMTP Login Process

After establishing contact with the SMTP mail server, we can start composing and sending the email. We'll first fire off an "eh" to the server. We won't delve too deeply into what this information conveys to the server. The next step is to send an authentication message to the server. The credentials saved in the variables "SMTP USER" and "SMTP PASSWORD" will be sent along with this login request.

smtp_server.help()

    smtp_server.login(SMTP_USER, SMTP_PASSWORD)

How to Layout an Electronic Message

The next thing we need to do is compose the message sent via the SMTP connection. This is crucial since it allows our Raspberry Pi to send you an alert if it detects a change to the website you've instructed it to monitor. This structure is adaptable to your needs. To avoid any confusion, please use the "FROM," "Too," and "Subject" lines as shown here. On different lines, please. In addition, one blank line should precede the main body of your email.

email_text = \

"""From: %s

To: %s

Subject: %s

%s

""" % (SMTP_FROM_EMAIL, SMTP_TO_EMAIL, subject, message)

Email transmission

As the final step of this procedure, we transmit the email over our SMTP session. The previously generated email, along with the email address from "SMTP FROM EMAIL" and the destination email address from "SMTP TO EMAIL," are passed into this function call. After the message has been transmitted, we terminate the SMTP session.

smtp_server.sendmail(SMTP_FROM_EMAIL, SMTP_TO_EMAIL, email_text)

    smtp_server.close()

Including Function Calls for email notification()

Since the necessary method has been written, it must be called from within the program. Here, we'll modify the "main()" procedure. You need to locate the line below in your Python code.

print("Non 2XX response while fetching")

Follow that with the line below. The script will automatically send you an email to notify you every time the RPi web monitor encounters an error connecting to the webpage.

Moreover, we should have a line that alerts us whenever the webpage updates. Check your script for the following line. Ideally, it would be located below the previously discovered line. Try to find the following:

The following line should be added above it. Every time this line is reached, an email will notify you that the web has been updated.

The Complete Code

After making the necessary modifications, your script's core code should resemble what we have provided as an example. Before moving forward, make sure to save your code. Your website monitoring service should now send you email alerts if functioning properly. It will send a notification email whenever it is launched and detects a change to the website. This website will, however, notify you by email if it becomes unavailable or produces a status code other than 2XX.

import os

import sys

import requests

from bs4 import BeautifulSoup

import smtplib


SMTP_USER='example@gmail.com'

SMTP_PASSWORD='password'

SMTP_HOST='smtp.gmail.com'

SMTP_PORT='465'

SMTP_SSL=True


SMTP_FROM_EMAIL='example@gmail.com'

SMTP_TO_EMAIL='sendto@gmail.com'


def email_notification(subject, message):

    "Send an email notification.


    Message - The message to send as the body of the email.

    """

    if (SMTP_SSL):

        smtp_server = smtplib.SMTP_SSL(SMTP_HOST, SMTP_PORT)

    else:

        smtp_server = smtplib.SMTP(SMTP_HOST, SMTP_PORT)


    smtp_server.ehlo()

    smtp_server.login(SMTP_USER, SMTP_PASSWORD)


    email_text = \

"""From: %s

To: %s

Subject: %s


%s

""" % (SMTP_FROM_EMAIL, SMTP_TO_EMAIL, subject, message)


    smtp_server.sendmail(SMTP_FROM_EMAIL, SMTP_TO_EMAIL, email_text)


    smtp_server.close()


def cleanup_html(html):

    """Cleanup the HTML content.


    html - A string containg HTML.

    """

    soup = BeautifulSoup(html, features="lxml")


    for s in soup.select('script'):

        s.extract()


    for s in soup.select('style'):

        s.extract()


    for s in soup.select('meta'):

        s.extract()


    return str(soup)


def has_website_changed(website_url, website_name):

    """Check if a website has changed since the last request.


    website_url - URL that you want to monitor for changes.

    website_name - Name used for the cache file.

    """

    headers = {

        'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; PIWEBMON)',

        'Cache-Control': 'no-cache'

    }


    response = requests.get(website_url, headers=headers)


    if (response.status_code < 200 or response.status_code > 299):

        return -1


    response_text = cleanup_html(response.text)

    

    cache_filename = website_name + "_cache.txt"


    if not os.path.exists(cache_filename):

        file_handle = open(cache_filename, "w")

        file_handle.write(response_text)

        file_handle.close()

        return 0


    file_handle = open(cache_filename, "r+")

    previous_response_text = file_handle.read()

    file_handle.seek(0)


    if response_text == previous_response_text:

        file_handle.close()


        return 0

    else:

        file_handle.truncate()

        file_handle.write(response_text)

        file_handle.close()

        

        return 1


def main():

    """Check if the passed in website has changed."""

    website_status = has_website_changed(sys.argv[1], sys.argv[2])


    if website_status == -1:

        email_notification("An Error has Occurred", "Error While Fetching " + sys.argv[1])

        print("Non 2XX response while fetching")

    elif website_status == 0:

        print("Website is the same")

    elif website_status == 1:

        email_notification("A Change has Occurred", sys.argv[1] + " has changed.")

        print("Website has changed")

        

if __name__ == "__main__":

    main()

Put your RPi web monitoring software on a schedule.

If you've taken the time to write a Python script to keep tabs on many websites, you'll want to set it to run regularly. You may implement an infinite loop to make the script run indefinitely; however, a cronjob will suffice. Our Python program must be installed on the RPi before you continue. You can grab the core code from the Git repo if you'd rather skip the intro. Be careful to complete the SMTP connection information. This instruction section assumes the script is located in the "pi" user's home directory (/home/pi/). After making the necessary changes to the crontab, our program will be run automatically every minute.

Start modifying the active user's crontab with this command.

crontab -e

When requested to choose a text editor, we suggest "nano." Please insert the following line at the end of the document. If you wish to track a certain URL, you'll need to make a small change to the command. In addition, a cache name must be specified. The cache name is optional and can be any string that helps you distinguish this request from others. Insert at the end.

CTRL + X, Y, and ENTER will save your changes to crontab. A new minutely check of the specified website will now be performed by your RPi. The software will notify you if it identifies a modification to a page's content.

Let's take a look at the main gains of monitoring your website.

  • Encourages problem-free operation of systems

To maintain optimal performance and satisfy clients, monitoring the website is essential. Customers are in a buying mood and don't want to wait for webpages to open or for controls that don't work. If you own an online store, you should examine key performance indicators such as daily bounce rates and page load time. However, regular usability testing will ensure consistent functionality and keep visitors returning.

  • Keeps a site running smoothly and efficiently

A growing top line is a sign of a healthy website. However, you will not be able to spot warning signs unless you understand your typical exchange rates and annual peaks. Monitoring your statistics can provide a clear picture of your website's performance.

  • Detailed user insights are revealed.

One of the essential benefits of web tracking is the ability to uncover hidden user and product possibilities. This may be achieved by setting up quarterly or monthly usability tracking based on your staff's population and your website's structure. One of the most effective strategies to enhance your site and raise customer satisfaction is demonstrating empathy for your customers by performing routine monitoring activities like analyzing comments.

E-commerce transactions can be optimized using customer feedback, for example, by removing an extra photo or description step between the product selection and the shopping basket. There's a chance that another user will vent their annoyance over not being able to access subscriber-only deals. If you take the time to listen to and empathize with your users' complaints, follow up with them to let them know they must sign in to obtain the offer and assure them that the site will be improved soon.

  • assures usability and boosts return on investment

Users may not be able to access and engage with your website in the event of downtime or page breaks. Prolonged outages can lead to being de-indexed, dropped in search engine rankings, and a dent in your reputation. No one will wait for a down website. Customers are not easily won back, and doing so requires an investment of both time and resources. If your site goes down in Perth but remains accessible in Sydney, you won't realize the difference unless you watch it constantly. In an emergency, tracking can assist you alert consumers and fixing any accessibility problems before they affect users.

In addition to preventing disruptions in the future, automatic downtime monitoring can provide real-time alerts by studying past incidents. Whether you need to check if your SSL certificate has expired or if there is an API problem affecting site speed, automation and control can help.

  • Facilitates easy interaction

When servers go down, your website becomes inaccessible, and so does all of your correspondence. Because of the prevalence of remote work and international teams, port failures have become increasingly expensive. Unfortunately, there is no viable or efficient way to communicate by text message on a mobile device.

Servers and port monitoring are often included in downtime monitoring software. Systems and tools make it easy to resolve communication issues between teams, offices, and nations. Customers will appreciate the timely alerts, and your employees will appreciate the mental stimulation.

Disadvantages of Website Surveillance

There are, of course, a few drawbacks to keeping an eye on your website. To decide if tracking is the best option for you and the team, consider these drawbacks. Some sites and portals may have a foundation of cybersecurity or automatic monitors; depending on your firm's specifics, you may just need a few more tools and processes. There is a silver lining to every cloud in the website monitoring storm. This article will provide some advice and solutions to help you deal with the drawbacks.

  • Costly

The cost of a good website monitoring technology might be a significant drain on a company's resources. Though many products provide free tiers of service, these tiers are sometimes not as feature-rich as paid tiers, leaving you vulnerable to security holes or unsupported APIs. Spending on paid monitoring software, if any, might range from $15 per month to $500 per month, depending on the features you require and the size of your firm.

  • Consumes a lot of time

Time is a necessary component of the organization, approach, and resource for successful web monitoring. How long would it take, for instance, to become an expert in website monitoring if you taught your current team how to do it? Because of the potential lack of workforce in other areas as a result of a single employee's pivot, training may not always be an option. Assume for a moment that your group possesses the necessary expertise to perform monitoring in-house. You should consider how much effort it will take and whether outsourcing is an option.

  • Blocklists

Checking in on your website regularly is essential. It's important to check in numerous times a day to ensure no outages. Regrettably, excessive monitoring can raise red flags with your hosting service, leading to your IP address being blocklisted. To some, blocklisting might prevent electronic communications and alert consumers that your site is a security threat, a major downside of website monitoring. However, this problem can be easily fixed by contacting your server and having your IP address allowed.

  • Monitoring errors

Even the most composed teams can quickly become flustered when confronted with novel tools and increased data. Mistakes in tracking can be caused by several factors, including a failure to communicate, inexperience, and hazy objectives. However, when a team is inundated with information, they may become paralyzed and unable to get the answers they need. Since the point of tracking is to swiftly handle problems within all data available instead of getting mired in data analysis, this may be particularly stressful and disheartening.

Conclusion

This tutorial taught you the fundamentals of setting up a web-based monitoring system on a Raspberry Pi 4. This code will retrieve the most recent version of the given URL each time it is used. Then, the cached response is checked again on the new request to determine if any changes were made. This script will notify you via an outside email server if the URL changes or becomes unavailable. In the following tutorial, we will learn how to build a raspberry pi 4 Wi-Fi extender for our home use.

Run Xbox Cloud Gaming on Raspberry Pi 4

Where To Buy?
No.ComponentsDistributorLink To Buy
1Raspberry Pi 4AmazonBuy Now

Introduction

Greetings, and welcome to the next section of our Raspberry Pi 4 tutorials. In the last section, we discovered how to set up and run our self-host bitwarden on our Raspberry Pi. We learned how to set up admin panels and perform a wide range of actions, such as limiting the creation of new accounts and users for security purposes. However, in this guide, we will discover how to configure a PS3 or PS4 joystick with our raspberry pi and set up and run Xbox cloud gaming on our Pi 4.

Components

  • Raspberry pi 4

  • SD card

  • Power supply

  • Ethernet cable or wifi

  • Xbox controller

  • Xbox game pass ultimate subscription

  • USB keyboard

  • USB mouse

  • HDMI cable

Through a service called Xbox Cloud Gaming, users may play a wide variety of games without having to download any software onto their devices. Here, we'll use Microsoft's Xbox Internet Gaming service to transmit these programs to your Pi 4. A membership to Xbox Gaming Pass Ultimate is required to use this feature. Xbox Gaming Pass is required for this streaming service, but the subscription unlocks a wealth of video game apps.

Incorporating Xbox internet Game streaming on Pi 4 is a breeze with the help of the Chromium internet browser. Remember that there will be some delay in responding to your inputs, but don't let that deter you from playing most games. As a bonus, this guide will show you how to convert your Raspberry Pi running the "light" version of the Xbox One operating system into a dedicated cloud gaming computer for the Xbox One.

Regarding game streaming services, Xbox internet Gaming isn't the only option for your Pi 4. Games can also be streamed to your Pi 4 using Google's Stadia program. Alternatively, you can use a streaming service like Steam to play your games on the go. To play games via Xbox Cloud, you'll need to be in a region where it's available.

Get your Raspberry Pi ready for setup.

You must fulfill a few prerequisites before you can begin using Xbox internet Gaming on your Pi 4. Here, you'll learn how to update your Raspberry Pi and set up the Chrome internet browser we'll need to connect to the cloud game platform. These instructions have been designed with Pi OS Lite compatibility in mind. The "Lite" version of Xbox internet Gaming will require additional effort.

To move on, we must ensure that our Raspberry Pi is up-to-date with the latest software. The software has to be updated so that we get the most incredible possible performance with the Xbox internet Gaming platform. You can upgrade your Pi's software to the most recent version using the following two instructions.

sudo apt update

sudo apt upgrade -y

After the update is finished, we can add any other software to our pi four that is required for Xbox internet Gaming. We'll primarily be using the Chrome internet browser, which can be obtained by running the instruction below.

sudo apt install chromium-browser xserver-xorg x11-xserver-utils xinit openbox bluealsa unclutter

We'll install just the bare minimum of software to get Chromium up and running on pi 4 OS Lite. Having a controller linked to your pi 4 is the next step. Most Xbox Cloud Gaming titles necessitate the usage of a controller. Despite the inclusion of third-party controller functionality, the Xbox joystick remains the superior option. And if you want to know which controllers work with their internet gaming platform, you can find that information in their official docs. Another option is to use a PlayStation joystick with Pi 4, and we will show you how to do just that.

How to Configure PlayStation joysticks on a pi 4

You may have discovered that using a PlayStation joystick with a Raspberry Pi is not a walk in the park. This tutorial is written in the hopes of making the whole thing easier.

Here at the pi 4 Wiki, we have a comprehensive tutorial for connecting and setting up Ps controllers of all generations. We walk you through installing the six-pair program for PS3 controllers and configuring your joystick to work with your pi four and any Bluetooth-enabled device. The procedure of connecting a PlayStation 4 joystick to a Raspberry Pi via Bluetooth is detailed here, and alternate methods are provided in case your joystick is incompatible with the default implementation of Bluetooth. The pi 4 is compatible with various game controllers, including Xbox joysticks.

Configure a PlayStation 3 Controller for Use with a pi 4

It's not easy to get a Ps3 controller working with a pi 4. If you want to use it wirelessly, we'll walk you through the setup process from start to finish. The wired PS3 joystick should function as simply a plug-and-play accessory. When using a PlayStation 3 controller, all of its functions must work correctly, and Sixad does just that. If you want to use a USB micro cable to connect the joystick to the pi 4, you should get one before you start this part of the pi 4 PlayStation tutorial. The configuration of the PS3 joystick for communicating with Wireless controllers is necessary for this.

Setup of the PlayStation 3 Joystick with the pi 4

The first step is to install a library, which will allow us to compile the six-pair code. The libusb-dev library enables the software to communicate with USB storage devices.

sudo apt install libusb-dev

Now that we have the necessary package loaded, we can get the six-pair script and set up a folder to store it.

mkdir ~/sixpair

cd ~/sixpair

wget http://www.pabr.org/sixlinux/sixpair.c

Now that we have the six pair code on our pi 4, we can build it with the help of the command below. Use this command to initiate a compilation using the GCC compiler.

GCC -o sixpair sixpair.c -lusb

Now that Sixpair has been compiled on our pi 4, we can connect our PS3 joystick to the RPi through its USB micro port. After connecting the joystick, use the following command to start six pair. So that our Wireless device can communicate with the joystick, Six pair will modify its settings.

sudo ~/sixpair/sixpair

If the six-pair program has successfully re-paired your PS3 joystick with your RPi's Wireless dongle, you will see the output in the terminal interface that looks like the example below.

After these modifications, you can disconnect the Playstation 3 joystick from the Pi 4; we won't need it plugged in through USB again unless you upgrade your Wireless adapter. You can either use the bluetoothctl program to communicate to the Playstation 3 joystick or compile and install sixad to manage the connection. We think you should give Wireless a shot because it works well with other wireless devices. Read on to learn more about utilizing Bluetooth to link your PlayStation 3 joystick. If you'd rather learn how to connect the joystick with SIXAD, you can do so below.

Getting your Playstation 3 joystick set up with bluetoothctl

Now that the Playstation 3 joystick has been prepped for use with the Pi 4, we can proceed to pair the two devices. The first step is launching the Bluetooth setup tool on pi four by entering the command below.

sudo bluetoothctl

Our Bluetooth setup tool is now active, and we can turn on the agent and tell it to use the default agent. To do so, simply enter the two instructions below.

agent on

default-agent

Following the successful execution of the preceding command, the program will begin scanning for new wireless devices, allowing us to locate them.

scan on

Any nearby wireless device will immediately become visible in the command prompt. Put these out of your mind for the time being and focus only on the Playstation 3 controller. The controller may now communicate with the pi four by holding the Ps button. The terminal's command line will begin to fill up shortly.

The MAC address will show in a format similar to the following; be sure to write it down. It's the string of words delineated by colons.

A MAC address has just become available, so write it down. Now that you have the MAC address, you may put it to use in the subsequent command. By issuing the command below, we hope to establish communication with the gadget and retrieve its unique identifier. You may need to retry this command several times before it works if you've tracked down a MAC address, substitute "YOUR MAC ADDRESS" with it.

connect YOUR_MAC_ADDRESS

If your Ps3 joystick stops attempting to connect, please check the status and hit the Playstation key again. If the following text shows in the cli, the connection was established successfully. We can go on to the next phase now that we have the UUIDs.

We need the MAC address to add the device to our Bluetooth-approved approved list. This enables the controller to connect with the Raspberry Pi without human intervention. We can accomplish this by inputting this command into our Pi 4. Be sure to change "YOUR MAC ADDRESS" to the address you uncovered.

trust YOUR_MAC_ADDRESS

After adding your Playstation 3 controller to the authorized list, the terminal should read as follows.

We may now exit the Bluetooth settings tool on the Pi 4 since the Playstation 3 joystick has been successfully paired with the Pi 4. Type the following commands or press CTRL + D to exit the program.

Quit

The RPi can be restarted at this point. This is done to verify that our system performs as expected during testing. To force the Raspberry to restart, type the command below.

sudo reboot

After rebooting the Pi 4, you can verify that your joystick appropriately communicates with the device by pressing the Playstation button. The controller's lights will momentarily blink when it connects, but subsequently, assuming everything went smoothly, only one morning will stay on.

Getting Your Playstation 3 Controller to Work with SIXAD

We need to install some prerequisite software on our RPi before we can begin compiling and configuring SIXAD. libusb-dev and libbluetooth-dev are required for compiling the sixad program and are essential libraries. Libusb-dev is a collection of source code that enables us to communicate with USB devices connected to a computer. To communicate with the Wireless stack in an OS, Libbluetooth-dev supplies the necessary code. In sixad's setup, this is what monitors for the presence of a Playstation 3 controller when it is plugged into the Pi 4.

sudo apt install git libbluetooth-dev check install libusb-dev

sudo apt install joystick pkg-config

Since we have everything we need, we can download the SIXAD repo from Retropie. We're using Retropie's fork because it includes some bug fixes and has been validated on the Pi 4. You can download the GIT repo onto our Pi 4 using the two commands below.

cd ~

git clone https://github.com/RetroPie/sixad.git

Now that we have the repo downloaded to our Pi 4, we can begin compiling. The following two instructions are all that are needed to compile SIXAD. The first line of code changes directories, and the second line of code builds the program by executing the makefile.

cd ~/sixad

make

Compiling the sixad code was the first step, but there are still a few things to accomplish before we can put it to use. The first step is creating a directory in which sixad stores all its user profiles. Create a new folder by running the command below.

sudo mkdir -p /var/lib/sixad/profiles

Let's finish by launching the checkinstall package we set up in the previous section. This program just verifies the current installation and, if necessary, executes the "make install" commands.

sudo checkinstall

During installation, you may be prompted to provide information or confirm selections; however, you can safely skip them and hit ENTER to proceed. Following completion of the installation, the following lines should show at your terminal.

The command below will launch sixad on your Raspberry Pi after completing the installation process.


sudo sixad --start

You should be prompted to push the Playstation key on your joystick to ensure proper operation. Connect the joystick to the Pi immediately.

Since we have sixad functioning, we need to set it the default loader at startup so we can always use our Playstation 3 joystick. The command below will insert sixad into the RPi's boot procedure.

sudo update-RC.d sixad defaults

How to Configure a PS4 Controller for Use With a Pi 4

The Ps4 controller can be used with the Pi 4 in various ways. There are a few of these that are easier to handle than others.

Wireless USB Adapter from Sony

To connect your PlayStation 4 joystick to your Pi 4, the Sony Bluetooth Adapter is your best bet. Especially considering that, in theory, the dongle should work immediately upon removal from its packaging and connect with any of the approved Playstation 4 controllers.

Bluetooth

To use Bluetooth, you'll need a Pi 4 or a USB wireless adapter in addition to the certified Sony Wireless adaptor. Although this solution should work for most Playstation 4 controllers, some users have reported success with a userspace gamepad driver named ds4drv. If you have already installed the SIXAD utility, you will need to remove it before you can use your Playstation 4 joystick with the Pi 4.

Universal Serial Bus Cable

To use your Playstation 4 joystick, simply plug it into a USB port on your computer using the same USB connection you used to connect to your Playstation 4. The Playstation 4 joystick should automatically connect via USB and work without additional setup.

Setting up BluetoothCTL for your Playstation 4 Controller

The next instructions are only necessary if you have followed the tutorial on pairing a Playstation 3 controller with a Pi 4 via Bluetooth. In that case, you can skip this part of the process to uninstall SIXAD.

The PS4 controllers cannot pair or function appropriately with SIXAD installed, so they must be deleted. Simply running the command below on Pi 4 will remove the compiled and configured sixad program we discussed in the PS3 part.

sudo dpkg -r sixad

Before setting up the rest of the system, let's double-check that our Pi 4 has the most recent updates by issuing the instructions below.

We will now start connecting our Playstation 4 controller to the Pi 4. The Bluetooth CLI utility will be used throughout this tutorial. Start up your RPi by entering the command below.

sudo bluetoothctl

After starting the bluetoothctl utility, we may activate the client and return it to its default settings by entering the commands below.

agent on

default-agent

We may now begin scanning for devices after activating the agent. Scanning must be started so that our controller appears when we initiate synchronization.

scan on

Now is the moment to simultaneously hit the "Share" key and the Playstation button, as depicted in the following diagram. Continue holding them until the joystick's light begins blinking.

If you see the output in the cli similar to what is shown below, jot down the MAC address. If you take the hex value after the ':' symbol, you have the MAC address.

If your joystick is still blinking, enter this command as soon as possible. When entering the MAC address, ensure to replace "YOUR MAC ADDRESS" with the actual value. It establishes communication with your Playstation 4 controller when you enter this instruction. Press the "Share" and "PS" keys to make the joystick stop blinking.

connect YOUR_MAC_ADDRESS

The following will display in the terminal window if the connection is successful.

So that the PlayStation 4 controller may instantly link to the Pi 4, we must now include our MAC address in the authorized list.

trust YOUR_MAC_ADDRESS

Now that the Wireless dongle on the Raspberry Pi recognizes the Playstation 4 joystick, we can exit the program by typing "quit." The next step is to verify the joystick's functionality.

Even though the joystick feature may already be present in your Raspbian distribution, let's install it. This set has the software we need to ensure our joystick is functioning correctly. You may get the program installed on your RPi by executing the following Unix command.

sudo apt install joystick

Once the joystick library is installed on your Pi 4, you can check js0 and retrieve its values by using the command below.

sudo jstest /dev/input/js0

Changing values whenever you move and push buttons on the joystick indicates that your Playstation 4 joystick is ready to use. However, you may need to resort to the userspace drivers if your joystick stubbornly refuses to function. Finally, after getting the joystick set up, we can move on to installing the cloud on the pi four os Desktop.

Online Xbox Play with Raspberry Pi Desktop OS

Gaming on the go with Xbox internet Gaming is a breeze with the Pi 4 OS Desktop and Microsoft's cloud service. Instead of devoting your computer solely to Xbox internet Gaming, you can use the desktop OS of pi 4. Chromium and iCloud can be accessed with the click of an icon and by entering the relevant URL, as shown below.

To access the internet with your RPi, launch the Chrome browser on the desktop. To access this, select the globe icon in your screen's upper right corner.

Open your internet browser and head to the Xbox internet Gaming website. To access this in Chrome, enter the following URL into the URL bar.

https://xbox.com/play

You'll need to sign in with your Xbox credentials if you haven't already done so from your Pi 4. You can access the login page by clicking the "Sign in" button.

You may see a "your browser is not supported" message after signing in. No need to panic; the XCloud server is compatible with the Chromium internet browser we are using on our Pi 4, so there's no need to switch browsers. To close this panel, click the "X" in its upper right corner.

Simply clicking on an Xbox game will begin streaming it to your Pi 4. Any gamepad, keyboard, or joystick can be used to navigate this interface.

There is a notice on the XCloud platform that says your streaming experience might not be ideal. Once you've selected a game for streaming to your Pi 4, you can keep playing by selecting the "CONTINUE ANYWAY" option.

You have now installed and configured the Xbox Internet Gaming platform to run on your Pi 4. Now that you know how to handle the joystick, you can have a blast playing the video game.

Playing Games over the cloud on an Xbox using the Pi 4 OS Lite

There are a few extra hoops to jump through to get the Xbox Internet Gaming platform working on a Pi 4 OS Lite device. After completing these procedures, your Raspberry Pi will automatically log you in and begin running Chromium.

Configuring Pi 4 for Automatic Login

If you wish to access Xbox Internet Gaming as soon as you turn on your Pi 4, you'll need to take the following settings. Skipping these will result in a login prompt for the "pi" user before you can use the internet browser.

We utilize the "Pi 4 Config tool" to set up our RPi so that it automatically enters the console at bootup, skipping the usual login process. The following command can be used to activate this utility on your gadget.

sudo raspi-config

Use the ARROW keys to move around the available options. To choose the highlighted item, use the Enter button. Starting with the raspi-config utility, select "System Options" from the drop-down menu. Select it by pressing the Enter button since it is the first choice.

You'll find "Boot / Auto Login" under "System Options" in the panel. Selecting this menu item may instruct the OS to log us into the console automatically.

The "Console Autologin" option comes next and must be chosen. If you select this, the Pi 4 will bypass the login screen and go straight to the console, where you can utilize the "pi" account.

Having made this selection, the RPi Config Wizard can be exited with the ESC key. When prompted, choose "Yes>," and then click the Enter button to restart the computer.

Setting up Chromium such that it launches alongside X-Server

Once the RPi is set up to enter the console at startup, the "autostart" file can be edited to initiate the Chrome internet browser. In addition, we need to edit the ".bashrc" file by adding a single line to ensure that the window utility Chromium needs to run launched automatically after the operating system has loaded.

So, let's begin by editing the "autostart" script in the "/etc/xdg/Openbox" folder. We'll use the nano editor to make the necessary changes to this script.

sudo nano /etc/xdg/openbox/autostart

Please copy and paste the lines below at the very end of this script.

xset s off

xset s noblank

xset -dpms

unclutter &

sed -i 's/"exited_cleanly":false/"exited_cleanly":true/' ~/.config/chromium/'Local State'

sed -i 's/"exited_cleanly":false/"exited_cleanly":true/; s/"exit_type":"[^"]\+"/"exit_type":"Normal"/' ~/.config/chromium/Default/Preferences

chromium-browser --disable-infobars --enable-features=OverlayScrollbar --kiosk 'https://xbox.com/play'

Our screensaver disable commands are located on the first three lines. We employ unclutter to conceal it merely as an alternative to entirely deactivating the mouse. This is helpful because we'll frequently use the mouse for things like logging in. Finally, we start Chrome on the RPi in kiosk mode, redirecting the browser to the Xbox Internet Gaming site.

After you've added these commands, save and exit the file. The final step is setting the Pi user's bash account so that the terminal automatically launches the window manager. 

nano ~/.bash_profile

This line has to be added at the very end of this document. This line will check whether it is the first time starting the terminal and if so, it will set the "$DISPLAY" property. When these prerequisites are met, the "startx" statement will be executed to launch the window manager.

[[ -z $DISPLAY && $XDG_VTNR -eq 1 ]] && startx

When you're finished adding that line, save and exit the file. If we want to use Chromium immediately, rather than after an RPi reboots, we can launch the window manager directly. When you launch the window manager now, Chrome will also launch because you modified the "autoboot" script earlier.

Startx

Be sure you're typing this command into the terminal of your Pi 4 itself. If you try to use it with SSH, you will be disappointed. If you choose, you can force a restart of your RPi by executing the command below.

sudo reboot

Utilizing Pi 4 OS Lite for Xbox Live Cloud Gaming

Once the RPi and Chrome have been set up, Xbox Live Games can be accessed. Your Pi's internet browser should automatically launch to the Xbox Game Pass portal.

To use Xbox Internet Gaming for the first time, you'll need to sign in using your Xbox credentials. To access the login page, please click the image below. In any case, the "Sign In" button on this page will get you started with the login procedure if you didn't notice it before.

After logging in, you may notice a message indicating that your browser isn't supported. Since this is a Chromium-based internet browser, the "X" in the corner can be used to close the current window.

We can now play games on our Pi 4 by streaming them through Microsoft's Live service with Xbox Game Pass. Make sure the controller is plugged into the Pi before choosing a game. Xbox Cloud platform also allows you to use your controller for menu navigation.

Here's a notification to let you know that the internet browser you're using isn't supported, which will appear whenever you try to play a video game that requires it. Selecting "CONTINUE ANYWAY" will allow you to disregard this notice safely.

We can play "Phoenix Point" through the Xbox Internet Gaming service, broadcast directly to our Pi 4 in the image below. The stream may not appear where you expect it to when it first begins. However, once play begins, things should correct themselves.


Conclusion

Now you should be able to play Xbox games through the internet on your Pi 4. Because it is included with Xbox Game Pass, Xbox internet Gaming allows you access to various games. To a large extent, this video game streaming platform runs smoothly on the Raspberry Pi 4. There may be spikes in latency now and then, but if you play video games that don't necessitate pinpoint controls, you can probably overlook them. The following tutorial will teach how to utilize Raspberry Pi 4 as a radio streaming and broadcasting device.

Self-Host Bitwarden In Raspberry Pi 4

Following this, we will configure bitwarden and host it on our Raspberry Pi 4. The last tutorial discussed utilizing a Raspberry Pi to install and run zeroTire on pi 4. By the end of the project, you will have learned how to set up a Raspberry Pi 4 with the necessary software for password management, including bitwarden, docker containers, and portainer, and how to configure their respective user interfaces.

Where To Buy?
No.ComponentsDistributorLink To Buy
1Raspberry Pi 4AmazonBuy Now

Why Would You Do This?

An effective password manager is a must-have. For the past decade-plus, I've relied on a password manager. The catch is that not all security-focused apps are created equal. Initially, I relied on password managers in my browser, but I soon switched to KeePass. It was an intelligent move ten years ago. The export features of my browsers allowed me to import all of my passwords into a single KeePass database, where I could then construct a system to keep track of them. However, technology has advanced, and now there are safer alternatives to these password managers. Once I discovered I could self-host Bitwarden, I felt it was my best option. I had been using LastPass with Mobile, but as they introduced sync limits into their free tiers, I decided to uninstall it. YubiKey compatibility is one of the features that Bitwarden lacks compared to LastPass, but which I was able to implement thanks to self-hosting. KeePass's lack of support for statistics on hacked, weak, and recycled passwords is disappointing. Most people agree that Bitwarden is the most excellent free password manager.

Components

  • Raspberry pi 4

  • Power supply

  • Ethernet or wifi

Preparing your Raspberry Pi for Bitwarden

To host our Bitwarden server on our domain, we must first take care of a few technical details. Setting up Docker is a necessary first step.

If we like, we can also decide to use Portainer for Docker container management. Either we can run it from the command-line interface or incorporate it into Portainer. A Bitwarden container can be managed with Portainer's user-friendly online interface.

We can then upgrade our operating system once Docker has been installed and a decision has been made regarding using Portainer. After applying the latest updates, our Raspberry Pi should be in peak running condition and ready to host Bitwarden.

Installation of Docker on a Raspberry Pi 4

It's no exaggeration to say that Docker is a powerful piece of software because it enables OS-level virtualization to distribute software applications within containers. When running inside a container, the software is restricted from accessing any resources that weren't explicitly allocated by the Docker runtime. Since everything is contained within the containers the runner installs, Docker makes it easy to distribute your software to devices. What's more, Docker accomplishes all of this with minimal extra effort. The software's low overhead makes it possible to function on a Raspberry Pi, which has little RAM and processor power.

The Docker team has made it easy to set up their container program by creating a handy installation script. Connecting via SSH to your RPi is an option for completing the tasks below.

The following command will fetch and run Docker's official installation script.

curl -sSL https://get.docker.com | sh

The code will be inserted into the command prompt if you use this command. Usually, you shouldn't, but you can trust Docker, so we'll make an exception here. You can view the script at get.docker.com if you are hesitant to run it before inspecting it. Time should be allowed for this code to finish as it will autodetect and install all requirements to run the Docker container on the RPi.

Portainer setup on a Raspberry Pi 4

The container management tool Portainer is small, accessible, and open-source. You may use this program to efficiently build, manipulate, and remove Docker containers on your Raspberry Pi. The software is simple to operate and requires no configuration outside of a Docker container, making it a breeze to set up.

Due to its lightweight design, Portainer doesn't significantly slow down your computer's performance. It's the best option for controlling your containers without messing with the command-line interface.

Once Docker has been installed and configured, Portainer can be installed on the Raspberry Pi. The newest version of Portainer can be obtained by using the command below, as it is hosted as a container on the public Docker hub.

sudo docker pull portainer/portainer-ce:latest

The docker file will be downloaded to your device after running this command. We tell it to get the ARM versions of the container by adding ":Linux-arm" to the conclusion of the pull request.

To begin using Portainer, wait for Docker to complete downloading the file to your RPi. We need to supply some more parameters when telling Docker to launch this container. Launch Portainer on your Pi by typing the command below into the console.

sudo docker run -d -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest

The most important things we accomplish here involve defining which ports Portainer should have access to. For us, it means using port 9000. Assigning the name "portainer" to this docker container facilitates easy recognition in the event of an emergency. If this Docker goes offline for whatever reason, the Docker management is instructed to start it up again.

Accessing Portainer with its Web-Based UI

You must establish a connection to the program's graphical interface before we can proceed with any tasks. You'll need access to an internet browser and the IP address of your RPi to complete this. The local IP address of your Raspberry Pi may be easily retrieved if you have forgotten it. The IP address allotted to our Pi locally can be shown by entering the hostname command.

hostname -I

For the quickest and easiest access to Portainer, simply type the following into your browser. You see that the port ":9000" is explicitly specified at the end of the address. The location at which Portainer can be accessed is this port.

http://[PIIPADDRESS]:9000

Always change "[PIIPADDRESS]" to your RPi's private IP address. This IP address was part of the previous steps' necessary actions.

How to set up Portainer on Pi 4?

When you first access the online interface for Portainer, you'll be prompted to make an administrator account. Specifying a username is required to establish this administrator account. Next, create a password for the new version. Passwords for Portainer must be a minimum of 12  letters in length. After entering the desired username and password, you can create the user by clicking the "Create user" button.

Now we can tell Portainer what kind of container ecosystem to maintain. The Docker option is the best fit for our needs. After making this selection, hit the Connect button.

The Portainer installation on your RPi should now be complete. Now you may use it to control the containers on any computer or mobile device.

Raspberry Pi 4 Bitwarden Setup

Now that our RPi is ready, we can proceed with installing the Bitwarden application. The availability of Bitwarden RS as a container in Docker makes its setup a breeze.

To set up Bitwarden with Portainer

The first part of this section will explain how to set up Bitwarden on Pi 4 using the Portainer online interface. We've already established that Portainer is an excellent option for managing Bitwarden on your RPi without using Secure Shell.

Bitwarden Container Preparation

First, we'll need to have Portainer set up so we can deploy the Bitwarden container. A few simple steps are required, but the total time is relatively short.

To get started, launch the Portainer web UI. The IP address of your Pi 4, followed by port 9000, will give you access to this.

Make sure that "[PIIPADDRESS]" is changed to the IP address of your Raspberry Pi.

You must switch to the local Container endpoint while loading the interface. To access the container management interface, please go here.

It is necessary to construct a volume for the Bitwarden containers before they can be created. Look for a link labeled "Volumes" in the menu slides out from the side.

There should be a collection of volumes you've made available here. An "Add volume" link must be placed here first. To access the volume settings, press the button.

A title should be selected for this new volume. In this tutorial, we will continue using Bitwarden exclusively. Simply give the book you're creating in RPi Bitwarden a title, and then hit the "Create the volume" option.

Following successfully creating the volume, we must now navigate to the "Containers" section of the interface. To toggle to this view, select "Containers" from the sidebar.

The containers that can be used with your RPi are listed below. The "Add container" button is up here somewhere.

Developing the Pi 4 Bitwarden RS Container

We have completed the preliminary setup and are ready to install the Bitwarden on our RPi.

As a first step, identify the container's label. Our own container's name is Bitwarden. The next step is to tell Docker which image to retrieve. This is the RPi's Bitwarden container in our instance. Don't forget to type "vaultwarden/server:latest" into the "Image" field.

The container's network configuration comes next. Find the heading labeled "Network ports setup."

Select the "publish a new network port" button below this heading twice. The boxes to set the Bitwarden ports to expose should now be visible.

Firstly, ensure the host is set to "127.0.0.1:8080" in the first box. Make sure that port 80 is entered into the container settings as well. From this port, you can access Bitwarden's web-based administration console.

To proceed to Step 2, please change the host to "127.0.0.1:3012". The correct port number for "container" is 3012. (3.). This port is used by Bitwarden for communicating over web sockets. As a side note, both host ports have been tied to the local computer. Because we can use them without any extra hardware, these are made public via a reverse proxy hosted by NGINX.

Keep going down until you reach the advanced containers settings heading. If you want to adjust the volume, click the "Volumes" tab. To add the RPi's volume to the container, we must next select the "map extra volume" option. Ensure that "/data" is entered into the "container" field.

At this point, we'll want to apply the custom volume setting we made before. Bitwarden-local is a good name for this.

Our next step is to keep Bitwarden running on the Raspberry Pi. Changing the startup policy will accomplish this. Simply select the "Restart policy" tab here. There should be an "Always" option for restarting. That way, even if the container goes offline for any reason, Docker will attempt to keep it running.

We may now deploy the Bitwarden containers after we have done setting it. The "Deploy the container" button may be found under this site's "actions" heading.

In the containers list, you'll find Bitwarden once Portainer has downloaded the file to your Pi 4. Below is a snapshot of our containers list after Bitwarden was successfully installed.

Docker command line interface installation of Bitwarden

You can also use the command line interface if you avoid installing and using Portainer with Bitwarden on the RPi. Following these instructions, you can successfully download and install Bitwarden on your device.

We will use Docker to retrieve the most recent version of Bitwarden RS. The newest server version will be downloaded and made available after these procedures.

docker pull vaultwarden/server:latest

Docker will automatically begin downloading Bitwarden RS to the Pi 4. The next step will be to launch the image. To accomplish this, please type in the command below and hit enter.

sudo docker run -d --name bitwarden \

    --restart=always \

    -v /bw-data/:/data/ \

    -p 127.0.0.1:8080:80 \

    -p 127.0.0.1:3012:3012 \

    vaultwarden/server:latest

With this command, our Bitwarden RS servers will begin running from the downloaded file. The ports to which we require Docker to connect the Bitwarden image are then specified. Here, we're listening for connections on port "8080" to our web server. As a next step, we open up "port 3012," which is the port Bitwarden's URL sockets use for communication. The RPi's localhost IP address will be the only one able to access these ports (127.0.0.1). In the following part, we'll set up a proxy server to enable HTTPS for Bitwarden, allowing it to be accessed from outside the local network.

Installing Bitwarden with an NGINX Proxy

We have Bitwarden functioning, but it is not usable unless we configure HTTPS. The Bitwarden WebClient uses secure HTTPS connections for certain JavaScript operations. Setting up an NGINX proxy is required to enable HTTPS functionality. NGINX will handle processing requests to our Bitwarden server hosted on an RPi.

Pi NGINX Preparation

Setting up NGINX is a prerequisite to deploying Bitwarden. To establish a secure HTTPS connection, it is necessary to install the web server program and create an SSL certificate.

NGINX installation is a prerequisite for continuing with this section. So then, let's go ahead and do it.

Raspberry Pi NGINX installation.

There are a few reasons why Nginx is a good choice than Apache for Pi 4. NGINX is preferable to other web servers for the Raspberry Pi because it makes more efficient use of the Pi's limited resources, such as its memory and processing power. While NGINX has more space to maneuver than Apache, you still shouldn't count on it to handle heavy loads or major PHP tasks.

Since Apache2 may already be installed on your machine, we will use the command below to remove it. Since we want to use NGINX as our web server, we have decided to disable it from autostarting and accessing port 80. If you know that Apache2 hasn't been set up on your Pi 4, you can ignore this step.

sudo apt remove apache2

Now that we have the most recent versions of all the packages and have gotten rid of Apache 2, we can continue with the procedure. Finally, use your RPi to execute the command below and set up NGINX.

sudo apt install Nginx

Now that NGINX has been installed, we may launch it. To activate your RPi's web server, enter the command that follows into the terminal.

Sudo systemctl start Nginx.

Now that the NGINX server is running, we may obtain a local IP address. This is done so that we can use a different web browser to check the functionality of our web server. Use the hostname statement to learn your RPi's local Internet Protocol address.

hostname -I

Now that we know the local Internet address for our Pi 4, we can visit that location in any internet browser. Once you have the local Internet Protocol Address by using hostname -I, simply navigate there. The address http://192.168.0.143 works for me.

http://YOUR PI's IP ADDRESS

If you type in the URL above, your browser should take you to the desired location. In the rare case that this displays an Apache page, you should not worry; NGINX does not always replace the Apache index.html file.

After NGINX is set up, we'll make an SSL cert for it. You can make a license on your RPi in two distinct ways.

You can use "Let's Encrypt to generate a trusted SSL certificate" if you have a domain name registered with them. A signed certificate cannot be issued with just an Internet address as the DNS name. If you follow this way, remember to write down the path to where your certificate is kept.

Secure Sockets Layer Certificates for Raspberry Pi with Let's Encrypt

Using this Certbot client and your existing web server or a temporary server, you can acquire an SSL cert from Let's Encrypt. To purchase an SSL Cert for your RPi, you must have a DNS name directed to your Internet address. Ensure your DNS settings are configured to avoid going through Cloudflare's proxy if you use them as your DNS provider. As the proxy disguises your actual internet protocol address, the Let's Encrypt program will be unable to validate your RPi's IP address and provide an SSL cert.

Now, using one of the commands below, we may install the LetsEncrypt application onto our Pi 4. "Cabot" is the program's name in question. Set up the certbot package for Apache if you're using that web server, or use the standalone certbot program.

sudo apt install python3-certbot-apache

sudo apt install certbot

Now that Certbot is set up, we can go to Let's Encrypt and request an SSL cert for our Pi 4. The situation can be approached from two different angles. You can ignore this step if you do not use Apache. If you're using Apache, you may quickly and easily install a certificate into Apache's settings by executing the following command. To begin, you must ensure that ports 80 and 443 are routed. Cloudflare, which disguises your Internet address, must be temporarily bypassed if it is your DNS service provider.

sudo certbot --apache

We can take two approaches to obtain a license cert from Let's Encrypt if Apache is not used. With certbot, we have the option of running a separate python server to seize the host machine. You can also use another web server, such as NGINX, and we will still be able to retrieve the certificate from it. Once you get the license, though, you will need to set it up manually.

If you want to use the web server without having to install anything more, all you have to do is ensure port 80 is open and forwarded. You must modify example.com to your domain name.

sudo certbot certonly --standalone -d example.com -d www.example.com

More expertise is needed to use webroot than the installed web server. Check that the directory /var/www/example exists and can be accessed from the internet by changing its pointing to the correct location.

sudo certbot certonly --webroot -w /var/www/example -d example.com -d www.example.com

Following these instructions, you will be asked to supply information, including your e-mail address. The information you provide will help Let's Encrypt maintain track of the licenses it issues and get in touch with you if any problems arise. After you enter the necessary data, it will immediately retrieve the permit from Let's Encrypt. If you are having problems, check that ports 80 & 443 are not banned and that your DNS name is correctly pointed to your IP address. Finally, if Cloudflare is your DNS service provider, double-check that the DNS settings currently exclude the proxy servers. This is where certbot client licenses will be saved after being downloaded. 

The secret key file for the license (privkey.pem) and the entire certificate chain (fullchain.pem) can be found in these directories. Keep in mind that these documents are what authenticates your Https connection and keep it safe; therefore, you shouldn't share them with anyone.

Building a solid Diffie-Hellman group is our final preparation step. This is to assist in strengthening SSL connections on your devices.

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

NGINX Proxy Configuration

To have NGINX act as a proxy for Bitwarden, a virtual host must first be created. Thankfully, setting NGINX as a proxy requires no effort.

First, we need to get rid of NGINX's factory settings file. If you intend to use Bitwarden with a domain name, there is no need to do so.

sudo rm /etc/nginx/sites-enabled/default

A newer NGINX config file must be made. Launch the nano editor and type the instruction below to get started editing this new configuration file.

sudo nano /etc/nginx/sites-enabled/bitwarden.conf

Copy and paste the below lines of text into this document. The first rule we're creating will force all HTTP (port 80) traffic to use HTTPS instead (port 443).

server {

    listen 80;

    listen [::]:80;

    server_name _; #Change this to your domain name

    return 301 https://$host$request_uri;

}

The web server block will manage the proxy and Hypertext transfer protocol connection we'll add next. If you created the certificate with Let's Encrypt, you'd also need to change the locations of the "SSL certificate" & "SSL certificate key" environment variables.

server {

  listen 443 ssl http2;

  server_name _; #Change this to your domain name

  

  ssl_certificate      /etc/ssl/certs/nginx-bitwarden.crt;   #Swap these out with Lets Encrypt Path if using signed cert

  ssl_certificate_key  /etc/ssl/private/nginx-bitwarden.key; #Swap these out with Lets Encrypt Path if using signed cert


  ssl_dhparam /etc/ssl/certs/dhparam.pem;


  # Allow large attachments

  client_max_body_size 128M;


  location / {

    proxy_pass http://0.0.0.0:8080;

    proxy_set_header Host $host;

    proxy_set_header X-Real-IP $remote_addr;

    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header X-Forwarded-Proto $scheme;

  }

  

  location /notifications/hub {

    proxy_pass http://0.0.0.0:3012;

    proxy_set_header Upgrade $http_upgrade;

    proxy_set_header Connection "upgrade";

  }

  

  location /notifications/hub/negotiate {

    proxy_pass http://0.0.0.0:8080;

  }

}

According to your needs, tweaks to your RPi Bitwarden server are sometimes necessary. If you have a custom domain name, you should substitute it for the server name.

At completion, your document should resemble the one displayed below.

You may now save this document. The NGINX service just has to be restarted at this point. Our modifications will not take effect until we continue the operation. Type in the following instructions if you want to reboot NGINX on your RPi.

sudo systemctl restart Nginx

The Bitwarden Web Client Access Procedure

The Bitwarden web application on our RPi is now accessible and usable, thanks to the installation of a proxy server.

Visit this link in your preferred web browser to access the Bitwarden interface. Always change "YOURPIIPADDRESS" to reflect your RPi's actual Internet address. A domain name, if you have one, must be used instead.

You will get a warning if you try using a certificate you signed yourself.

Creating a Bitwarden account is a prerequisite to using the service. There must be a "Create Account" link on the login screen. If you can locate this link, please click it.

Starting immediately, you can enter data for your brand-new account. To get prompted, type in your Bitwarden account's e-mail address. You'll need to use this e-mail address to sign in. Next, provide Bitwarden with a label to utilize throughout its user interfaces. Next, you'll want to protect this account with a strong password. This needs to be a robust and difficult-to-guess password. After reading and accepting Bitwarden's terms of service and privacy policy, you can finish creating your account. When you're through customizing your account, click the "Submit" button.

Having signed up, you can now access your Bitwarden vault. Enter the e-mail address you used to create the account first. Now, please log in using the password you just created. The "Log In" button is active, allowing you to enter the site.

Once you've installed Raspberry Pi Bitwarden, you may begin using it as a secure repository for your data.

Allowing access to the Bitwarden admin Panel

Your account has been created, and the administrator token can be created now. You'll need the admin token to go into the Bitwarden control panel. We'll have to adjust the settings on our Docker containers to accommodate this. You will have access to a list of all users who have registered, as well as the option to delete them from the administrative dashboard. Even if you've turned off the feature, you can still create invites for new users. Bitwarden's settings can also be adjusted through this user interface. You may wish to decide if you want others to be capable of signing up.

Create a new Admin Token

The first order of business is to produce a new Bitwarden administrator token. Given this token's importance, we will use OpenSSL to construct a long string of robust randomly generated digits. You may create this secure code using the following program on your Pi 4.

openssl rand -base64 48

Keep this token secure as it provides unrestricted access to the Bitwarden RS server if it falls into the wrong hands.

Connecting to the admin Panel

This current admin token will need inputted into the Bitwarden Docker container running on our RPi. The Portainer online interface or a customized command line can be used to accomplish this.

Configuring the Admin Token in Portainer

The Portainer UI must be launched, and then the container list revisited to change the administrator token. Find the Bitwarden container in the list of containers. Click on the link once you've located it to read more about it.

There must be a "Duplicate/Edit" button at the top of the page you're about to visit. The container's properties can be modified by clicking this button.

To access the "Advanced container options," please continue scrolling down. Click the "ENV" button to switch to the Environment tab under this header. For us to add the administrator token, please select the "add environment variable" button. After you click the button, you should see two new text fields appear at the page's footer. Just enter "ADMIN TOKEN" into the "name" field. A valid admin token must be entered in the "value" field. You can now click "Deploy the container" after creating the administrator token.

Please read the following message before attempting to update the Bitwarden container on your Raspberry Pi using Portainer. The container with the same name will be replaced, as shown by this message. For this, select the "Replace" option.

Configuring the Admin Token via the Terminal

Using a command prompt to update a container is a little more involved than doing it via the GUI, as you will need to delete the old container by hand. The Bitwarden container on our Pi 4 will need to be taken offline for us to delete it. Using this command, you can terminate a container that is already executing.

sudo docker stop bitwarden

Next, the old container must be discarded. If a container with an identical name and port number already exists, Docker will not allow us to create a duplicate. Execute the following command to get rid of the current Bitwarden container.

sudo docker rm bitwarden

The final step is to restart the docker process. Instead of using the default token, we'll use the administrator token we just created.

sudo docker run -d --name bitwarden \

    -e ADMIN_TOKEN=EXAMPLEPIMYLIFEUPADMINTOKEN \

    --restart=always \

    -v /bw-data/:/data/ \

    -p 127.0.0.1:8080:80 \

    -p 127.0.0.1:3012:3012 \

    vaultwarden/server:latest

To use these new commands, you must first produce a token and replace all occurrences of "EXAMPLETHEENGINEERINGPROJECTS" with that key.

The Bitwarden Control Panel Login

Access the Bitwarden administration website once you've successfully implemented your freshly generated administrator token on Pi's Bitwarden setup. You should type this address into your internet browser's address bar. Remember to change "YOURPIIPADDRESS" to the address of your Pi 4.

https://YOURPIIPADDRESS/admin

The Bitwarden control panel can be accessed at this location.

The created administrator token is required to be entered here. You can now log in by selecting the "Enter" button after you have provided the token.

To manage your Bitwarden account, you can now visit the control panel. These sections allow you to control who has access to Bitwarden and how it operates.

Prevent the creation of new users

After logging in for the first time and gaining access to the control panel, you may remove the option for new users to register. As a result, only anyone you grant permission can add new users to the Bitwarden safe.

Bitwarden's new account creation can be disabled in the admin interface of your Raspberry Pi. Once you have logged into the control panel as an administrator, you can continue.

New user registrations can be disabled using a setting in the system's global preferences. Click the corresponding button to access the locations buried in the "General settings" panel.

You should see an option labeled "Allow new signups" within the "General settings" menu. To turn off this feature, uncheck the box close to it.

A confirmation of the new settings will appear at the end of the page. There should be a blue "Save" button near the bottom of the page. To keep the current changes, please click here.

Conclusion

As of this point, your Raspberry Pi ought to be running Bitwarden. Clients for Bitwarden, a free and open-source password manager, are available for virtually any platform. Using a modified version of the standard client that uses fewer system resources, you may set up your Bitwarden web service on your Raspberry Pi. The following article will teach you how to use Pi 4 to play Xbox games in the cloud.

How to Build a Raspberry Pi FM Transmitter

Throughout our lives, we've relied on Radio and tv stations to keep us engaged. While we're on the subject of contradictions, it's also fair to say that these Stations can become tedious at times due to the RJ rambling on about nothing or annoying advertisements, and this may have left you wondering why you can't own a Radio station to broadcast your data over short distances.

Almost any electronics technician uses coils and other hardware to make an FM transmitter, although the tuning process is time-consuming and difficult. Setting up your FM station and going live in your neighborhood shouldn't take more than 30 minutes using an RPi. If you use the right antenna, you must be able to transmit to your school or community within 50 meters. Wow, that's interesting! So, let's get started right now.

Caution: This project is an education project and should not be abused in any way that might harm or inconvenience anyone. Interfering with neighboring FM frequencies is illegal, so please exercise caution when using this feature. In the event of any mishaps, we take no responsibility for them.

Where To Buy?
No.ComponentsDistributorLink To Buy
1Jumper WiresAmazonBuy Now
2Raspberry Pi 4AmazonBuy Now

Components

  • Raspberry Pi

  • Internet connection

  • Microphone

  • An enthusiastic RJ

Pre-requisites

Your RPi should already be running an os and be able to establish a network connection. If you haven't done so already, go through the instructions on how to use a raspberry pi.

A virtual server such as VNC, or a putty terminal window, is assumed to be accessible to you at all times. For the sake of this tutorial, we will run the program on RPi using the putty terminal session.

Frequency Modulation: What is it?

A method or procedure known as FM involves changing the carrier signal frequencies to match the frequencies of the modulated signal to encode information on a specific signal. Since data must be conveyed after being transformed into an electrical signal, a modulation signal is simply that.

A carrier's signal is transformed by an original signal in the modulation technique, which uses a methodology similar to amplitude modulation. On the other hand, FM maintains or maintains a steady signal loudness.

Why is FM necessary?

Fm is primarily used to decrease noise as well as the size of antennae, respectively. We know that a bigger antenna is required to send reduced frequency signals, whereas a smaller one is necessary to broadcast high-frequency signals.

Therefore, the sound signals are transformed into high-frequency radio waves and broadcast using the FM technique. Once more, the demodulation circuit on the receiver's side converts the high-frequency radio transmission frequency into the original understandable audio signal.

There is little interference since different signals are transmitted over a specific channel using separate wavelengths. So many folks can converse simultaneously and unhindered in a large metropolis.

FM transmission using coils

The construction of a long-range FM transmitter has long been on my bucket list of cool things. I've been so intrigued by some of the transmitter's uses, particularly since I was a kid, and spent much of my time fantasizing about how awesome it would be to have a few of the technology and technologies used in espionage movies. So lately, while reviewing one of my residence automation projects by using RPi and the motion package library, I felt it would be cool to add audio to the deliverable and stream live, so in addition to the multimedia feedback provided by the RPi, I could also get sound feedback out from area under monitoring. Even while this FM transmitter would not let me hear distantly (a range exceeding 10 kilometers), it will at least enable me to maintain an "ear" on events while I am about the property; then, after building it, I would have accomplished a few of the objectives that the younger me had set out to attain. It took me a few days ago, but I eventually got the motivation to make one, and I'll show you how to do it in today's post.

To avoid violation of policies of some countries, this experiment is being conducted solely for educational and scientific purposes. Keep the FM transmission at a low range and make sure it is built in compliance with applicable laws of one’s nation and therefore does not induce a disruption to others. This is essential. For any mishaps, I take no responsibility.

How do FM transmitters work

Using the concepts of the modulated signal, an FM transmitter can transmit the audio it receives from its input. Most FM transmitters are constructed in the manner depicted in the diagram below.

An amplification is frequently included in the transmitters because the transmission power of audio input is relatively low. This is done by utilizing an oscillator component to produce the carrier signal, which is then combined with an audio stream to generate a modulated signal that can be transmitted. When it comes to communicating, the low-impedance modulation signal is fed via a power amplifier to get to the antenna.

FM broadcast circuit

The electrical component should be connected as illustrated in the following FM transmitter diagram.

In this photo, you can see the prototype version of the FM radio transmitter.

The first transistors amplify the mic's output signals to a level suitable for transmission because the mic's output waveform is typically modest. In addition to amplification, the transmitter must also modulate. After that, the boosted audio signal is combined with the desired transmission carrier frequency to create a final signal. Because there is no visible output to identify the precise frequency where the transmitters are working, you may need to modify the FM transceiver radio well within the frequency range stated above to get the transmission frequency. This carrier signal can be differed using a 20pF capacitor attached to the inductor. The traditional spectral range of this specific design would be between 88MegaHeartz and 108MegaHertz. Once the carrier signal has modulated the audio signal, it is sent to the antenna, where it is received.

The resistors and capacitors used aren't set in stone, so you can experiment with them to get the best performance out of the transmitters.

Several other uses for this design aren't mentioned above, such as baby monitors or school address systems. Before constructing any of those practical items, please remember to check your local laws.

How Pi 4 works as a transmitter 

How can RPi, a board designed to serve as a development platform for microprocessors, do all of this? What if I don't need extra hardware to use the board as an FM station?

To prevent interference problems, each CPU will have a synchronized digital device. A signal known as a Spread-spectrum system clock, or SSCS is responsible for this Electromagnetic interference suppression. This frequency ranges from 1 MegaHeartz to 250 MegaHeartz, which fortunately fits well within the FM spectrum. We can make the Pi become an FM broadcaster by developing some code that uses the spread-spectrum clock frequency to modulate the frequencies. The Rpi Pi's GPIO pin 4 is where the frequency modulation will be sent. To use this pin as an antenna, we need a piece of standard wire attached to it no longer than 20 cm in length.

Getting the pi 4 readies for programming

Otherwise, read on for instructions on accessing your pi through the Command window if you haven't already done so. Boot Raspberry Pi with an HDMI connection to a display and an input device once you've installed a new operating system.

Link your Raspberry to the network by searching for a network option on the Raspberry desktop. Then go to the raspberry menu, click raspberry settings, and activate the SSH connection afterward. On your Windows or MAC computer, reconnect your computer to the same network as your Raspberry Pi so that both devices may communicate with each other on the local area network. You're ready to begin now that you've had Putty installed and running. Enter the Raspberry Pi's Internet protocol address and press enter. If you don't know your PI's Internet address, go to the admin side of your router and see if it's 192.168.43.XXX or something similar. An open command prompt will appear and ask for your login and passcode if all is done correctly. The default login and passcode are pi and Raspberry, respectively. Press Enter to see the next screen after entering it.

Convert Pi 4 into an FM station

GitHub provides the code needed to turn the Raspberry into a radio station. This page can be cloned directly into Raspberry; the application can be compiled and launched if you understand how to do so. Just follow the instructions below, and you'll be transmitting your audio files in no time.

Step 1:

Set up a new directory for our software files and put them there. Create a new guide by typing 'mkdir RPI FM' inside the command prompt, and then enter the folder using the word 'cd.'

mkdir PI_FM

cd PI_FM

Step 2: 

We must now copy the application from GitHub and place it in the folder we just made. Since we've previously moved inside the folder, we can run the following command below to complete the task.

sudo git clone https://github.com/markondej/fm_transmitter

Step 3:

We'll need a C compiler and other tools to run the C program we just acquired. GCC and G++ are the tools for this code, and the software for compiling them is termed make. To obtain compilers, enter the code shown below. Once the file is downloaded, your display will appear like the one below.

sudo apt-get install GCC g++ make

Step 4:

Compiling the code is now a cinch. You can do this by going into the folder using the change directory 'cd' FM transmitter and then compiling the script with root user 'sudo make .'The screen below should appear once your code has been successfully compiled.

cd fm_transmitter

sudo make

Step 5:

Launching the system is the last step. The intensity for which we would like to transmit, and the identity of the audio recording we would like to play must be specified when the program is launched. Star wars.wav is the default sound file that will be retrieved together with the code. We'll play the Movie Theme song at a 100megahertz frequency for testing purposes. The launch line's syntactic structure is:

sudo ./fm_transmitter [-f frequency] [-r] filename

The channel will be 100 MegaHeartz long because we need to play the movie file at that frequency.

sudo ./fm_transmitter -f 100  -r star_wars.wav

Test the Pi 4 FM transmitter

After you have started the application and you see the playback message as seen above, we may link an antenna to a Gpio 4 of Raspberry, I use a standard connecting wire, and it works perfectly for me.

Take a Radio, then set it to 100MegaHeartz channel, and you'll be able to listen to the movie music being aired. After making sure it works, you may switch out the movie theme with any other music or audio recording you choose and broadcast it with the same instructions as in step 5.

Transmitting live sound

While playing pre-recorded audio clips is entertaining, using this Pi 4 FM broadcast live audio would be much more enticing. With the help of the same tool, this is also possible. Just plug a mic into the Raspberry Pi's USB connection and modify the startup command-line interface. For additional information on this, please visit the GitHub homepage. Use the comment section of the forums if you run into any issues getting this to function.

Applications of FM

When it comes to frequency modulation uses, radio transmission dominates the list. Due to its higher signal-to-noise ratios, it provides a significant advantage in a radio broadcast. That is, there is little radio wave interference as a result. This is the fundamental justification for why so most radio stations choose to transmit music via FM.

Furthermore, many of its applications can be found in telematics, geophysical prospecting, EEG, various radio technologies, music creation, and devices used for broadcasting video. Fm offers a significant benefit over all other modulations in a radio broadcast. It will resist radio wave disruptions far better than an equally powerful modulation amplitude (AM) signal because it has a higher signal-to-noise ratio. The majority of music is aired through FM radio for this important reason.

  • Radio transmission frequently uses pulse modulation technology. Each radio broadcast station has its frequency range, and all broadcaster station signals are sent over the same transmission system. We can adjust the Radio's tuning to link it to a specific radio channel.

  • Our pc connections also employ pulse modulation technology.

  • The pulse modulation method is employed in magnetic storage tape recording systems.

  • Radio Detecting And Range (RADAR) systems employ the pulse modulation approach.

  • Multimedia content communications, including voice/video broadcasts, also use pulse modulation technology. Most of the time, the sound is delivered over FM, and occasionally, the film is as well.

  • The modulated signal generates an electrical impulse for usage in electronic instruments.

  • The monitoring system also makes use of FM technology.

  • Audio is synthesized by using the FM technology in pc sound adapters.

  • Military communication systems like Walkie-Talkies employ pulse modulation technology.

  • Additionally, Bluetooth and Zigbee communications technology utilize the FM method.

  • The Broadcasting method is also employed in ambulance systems.

  • The satellite radio technology uses FM technology.

  • Due to its low electronic noise, this FM method is employed in two-way radio transmission.

Benefits of FM

  • Low noise distortion

  • A smaller antenna is needed for pulse modulation equipment.

  • The pulse modulation platform's can be built to consume little power. This is a significant benefit of the modulation technique.

  • The pulse modulation process is more efficient because the signal's amplitude is always consistent.

Drawbacks of FM

  • The frequencies modulation circuit has many intricate parts.

  • A carrier wave is required for the frequency modulation process.

  • Amplification modulation is appropriate for long transmission lines, while FM is not.

Conclusion

In this article, we have learned how to create a radio station using a raspberry pi 4 with a few very simple steps. We have broadcasted a Star Wars movie theme through this system, and now you can try many other forms of data to broadcast, including video and live sound using a mic to get more familiar with the system. The next tutorial will teach us how to build a temperature log.

Stop Motion Movie System using Raspberry Pi 4

Thank you for joining us for yet another session of this series on Raspberry Pi programming. In the previous tutorial, we built a motion sensor-based security system with an alarm. Additionally, we discovered how to use Twilio to notify the administrator whenever an alarm is triggered. However, in this tutorial, we'll learn how to build a stop motion film system using raspberry pi 4.

Where To Buy?
No.ComponentsDistributorLink To Buy
1BreadboardAmazonBuy Now
2Jumper WiresAmazonBuy Now
3Raspberry Pi 4AmazonBuy Now

What you will make

With a Raspberry Pi, Py, and a pi-camera module to capture images, you can create a stop-motion animated video. In addition, we'll learn about the various kinds of stop motion systems and their advantages and disadvantages.

The possibilities are endless when it comes to using LEGO to create animations!

What will you learn?

Using your RPi to build a stop motion machine, you'll discover:

  • How to install and utilize the picamera module on the RPi

  • This article explains how to take photos with the Picamera library.

  • RPi GPIO Pushbutton Connection

  • Operate the picamera by pressing the GPIO pushbutton

  • How to use avconv to create a video clip from the command prompt

Prerequisites

Hardware

  • Raspberry Pi 4

  • Breadboard

  • Jumper wires

  • Button

Software

It is recommended that FFmpeg comes preconfigured on the most recent release of Raspbian. If you don't have it, launch the terminal then type:

sudo apt-get update

sudo apt-get upgrade

sudo apt install FFmpeg

What is stop-motion?

Inanimate things are given life through the use of a sequence of still images in the stop-motion cinematography technique. Items inside the frame are shifted slightly between every picture to create the illusion of movement when stitched together.

You don't need expensive gadgets or Graphics to get started in stop motion. That, in my opinion, is the most intriguing aspect of it.

If you've ever wanted to learn how to make a stop-motion video, you've come to the right place. 

Types of stop-motion

  1. Object-Motion

Product Animation can also be referred to as the frame-by-frame movement of things. You're free to use any items around you to tell stories in this environment.

  1. Claymation

Changing clay items in each frame is a key part of the claymation process. We've seen a lot of clever and artistic figures on the big screen thanks to wires and clay.

  1. Pixilation Stop Motion

Making folks move! It is rarely utilized. For an artist to relocate just a little each frame, and the number of images you would need, you'll need a lot of patience and possibly a lot of money, if you're hiring them to do so.

The degree of freedom and precision with which they can move is also an important consideration. However, if done correctly, this kind can seem cool, but it can also make you feel a little dizzy at times.

  1. Cutout Animation

One can do so much with cuts in cutout motion because of this. two-dimensional scraps of paper may appear lifeless, yet you may color & slice them to show a depth of detail.

It's a lot of fun to play about with a cartoon style, but it also gives you a lot more control over the final product because you can add your graphics and details. However, what about the obvious drawback? I find the task of slicing and dicing hundreds of pieces daunting.

  1. Puppet Animation

Having puppets can be a fun and creative way to tell stories, but they can also be a pain in the neck if you're dealing with a lot of cords. However, this may be a challenge for professional stop motion filmmakers who are not the greatest choice to work with at first. These puppets are of a more traditional design.

When animators use the term "puppet" to describe their wire-based clay character, they are referring to claymation as a whole. Puppets based on the marionette style are becoming less popular.

  1. Silhouette Stop Motion

Position the items or performers behind a white sheet and light their shadows on the sheet with a backlight. Simple, low-cost methods exist for creating eye-catching animations of silhouettes.

How long does it take to make a stop-motion video?

The duration takes to create a stop-motion video is entirely dependent on the scale and nature of your project. Testing out 15- and 30-second movies should only take an hour or two. Because of the complexity of the scenes and the usage of claymation, stop-motion projects can take days to complete.

Connect the camera to the raspberry pi.

You must first attach the camera to the Pi before it can begin rebooting.

Next to Ethernet, find the camera port. Take a look at the top.

The blue side of the strip should face the Ethernet port when it is inserted into the connector. Push that tab downward while keeping the ribbon in place.

Try out the camera

Use the app menu to bring up a command prompt. The following command should be typed into the terminal:

libcamera-hello

If all goes well, you'll see a sneak peek of what's to come. What matters is that it's not upside-down; you can fix it afterward. To close the preview, hit Ctrl + C.

For storing an image on your computer, run the command below:

libcamera-jpeg -o test.jpg

To examine what files are in your root folder, type ls in the command line and you'll see test.jpg among the results.

Files and folders will be displayed in the taskbar's file manager icon. Preview the image by double-clicking test.jpg.

There is no default way to make Python Picamera work with Raspbian newest version.

To make use of the camera module, one must activate the camera's legacy mode.

The command below must be entered into a command window:

sudo raspi-config

When you get to Interface Options, hit 'Enter' on your keyboard to save your changes.

Ensure that the 'Legacy Camera option is selected then tap the 'Return' key.

Select Yes using the pointer keys and hit the 'Return' key.

Repeat the process of pressing 'Return' to verify.

Click on Finish with your mouse cursor buttons.

To restart, simply press the 'Return' key.

Py IDLE can be accessed from the menu bar.

While in the menu, click Files and then New Window to launch a Python code editor.

Paste the code below paying attention to the capitalization with care into the newly opened window.

from picamera import PiCamera

from time import sleep

camera = PiCamera()

camera.start_preview()

sleep(3)

camera.capture('/home/pi/Desktop/image.jpg')

camera.stop_preview()

Using the File menu, choose Save Animated film.

Use the F5 key to start your program.

You should be able to locate image.jpg on your desktop. It's as simple as clicking it twice to bring up a larger version of the image.

It's possible to fix an upside-down photo by either repositioning your picamera with a camera stand or by telling Python to turn the picture. Adding the following lines will accomplish this.

camera.rotation = 180

Once the camera is set to PiCamera(), the following is the result:

from picamera import PiCamera

from time import sleep

camera = PiCamera()

camera.rotation = 180

camera.start_preview()

sleep(3)

camera.capture('/home/pi/Desktop/image.jpg')

camera.stop_preview()

A fresh photo with the proper orientation will be created when the file is re-run. Do not remove these lines of code from your program when making the subsequent modifications.

Connect a physical button to a raspberry pi

Hook the Raspberry Pi to the pushbutton as illustrated in the following diagram with a breadboard and jumper wires:

Pushbutton may be imported at the beginning of the program, attached to pin17, and the sleep line can be changed to use the pushbutton as a trigger in the following way:

from picamera import PiCamera

from time import sleep

from gpiozero import Button

button = Button(17)

camera = PiCamera()

camera.start_preview()

button.wait_for_press()

camera.capture('/home/pi/image.jpg')

camera.stop_preview()

It's time to get to work!

Soon as the new preview has begun, press the pushbutton on the Pi to take a picture.

If you go back to the folder, you will find your image.jpg there now. Double-click to see the image once more.

Take a picture with Raspberry Pi 4

For a self-portrait, you'll need to include a delay so that you can get into position before the camera board takes a picture of you. Modifying your code is one way to accomplish this.

Before taking a picture, put in a line of code that tells the program to take a little snooze.

camera.start_preview()

button.wait_for_press()

sleep(3)

camera.capture('/home/pi/Desktop/image.jpg')

camera.stop_preview()

It's time to get to work.

Try taking a selfie by pressing the button. Keep the camera steady at all times! It's best if it's already mounted somewhere.

Inspect the photo in the folder once more if necessary. You can snap a second selfie by running the application again.

Things to consider for making a stop motion animation

  1. You must have a steady pi-camera!

This is made easier with the aid of a well-designed setup.  To avoid blurry photos due to camera shaking, you will most likely want to use a tripod or place your camera on a flat surface.

  1. Keep your hands away from the pi-camera

If you don't press the push button every time, your stop-motion movie will appear the best. To get the camera to snap a picture, use a wireless trigger.

  1. Shoot manually

Maintain your shutter speed, ISO, aperture, and white balance same for every photo you shoot. There are no "auto" settings here. You have the option of selecting and locking the app's configurations first. As long as your preferences remain consistent throughout all of your photos, you're good to go. The configurations will adapt automatically as you keep moving the items, which may cause flickering from image to image if you leave them on auto.

  1. Make sure you have proper lighting.

It's ideal to shoot indoors because it's easier to regulate and shields us from the ever-changing light. Remember to keep an eye out for windows if you're getting more involved. Try using a basic lighting setup, where you can easily see your items and the light isn't moving too much. In some cases, some flickering can be visible when you're outside of the frame. Other times the flickering works well with animation, but only if it does so in a way that doesn't disrupt the flow of the project.

  1. Frame Rate

You do not get extremely technical with this in the beginning, but you'll need to understand how many frames you'll have to shoot to achieve the series you desire. One sec of the film is typically made up of 12 images or frames. If your video is longer than a few secs, you risk seeming like a stop motion animation.

  1. Audio

When you're filming your muted stop motion movie, you can come up with creative ways to incorporate your sound later. 

Stop-motion video

The next step is to experiment with creating a stop motion video using a collection of still photos that you've captured with the picamera. Note that stills must be saved in their folder. Type "mkdir animation" in the command line.

When the button is pushed, add a loop to your program so that photographs are taken continuously.

camera.start_preview()

frame = 1

while True:

    try:

        button.wait_for_press()

        camera.capture('/home/pi/animation/frame%03d.jpg' % frame)

        frame += 1

    except KeyboardInterrupt:

        camera.stop_preview()

        break

Since True can last indefinitely, you must be able to gently end it. If you use Ctrl + C to force it to end, the picamera preview will collapse and the loop will be terminated because it is using try-except.

Files stored as "frame" with a three-digit number preceded by a leading zero (009,005.) are known as "frame" files because of the % 03d format. This makes it simple to arrange them in the proper sequence for the video.

To capture each following frame, simply push the button a second time once you've finished rearranging the animation's main element.

To kill the program, use Ctrl + C when all the images have been saved.

Your image collection can be viewed in the folder by opening the animation directory.

Create the video

To initiate the process of creating the movie, go to the terminal.

Start the movie rendering process by running the following command:

FFmpeg -r 10 -i animation/frame%03d.jpg -qscale 2 animation.mp4

Because FFmpeg and Py recognize the percent 03d formatting, the photographs are sent to the movie in the correct sequence.

Use vlc to see your movie.

vlc animation.mp4

The renderer command can be edited to change the refresh rates. Try adjusting -r 10 to a different value.

Modify the title of the rendered videos to prevent them from being overwritten. Modify animation.h264 to a different file to accomplish this.

What's the point of making stop motion?

Corporations benefit greatly from high-quality stop motion films, despite the effort and time it takes to produce them. One of these benefits is that consumers enjoy sharing these movies with friends, and their inspiring content can be associated with a company.  Adding this to a company's marketing strategy can help make its product extra popular and remembered.

When it comes to spreading awareness and educating the public, stop motion films are widely posted on social media. It's important to come up with an original idea for your stop motion movie before you start looking for experienced animators.

Stop Motion Movie's Advantages

In the early days of filmmaking, stop motion was mostly employed to give animated characters the appearance of mobility. The cameras would be constantly started and stopped, and the multiple images would all be put together to tell a gripping story.

It's not uncommon to see films employ this time-honored method as a tribute to the origins of animations. There's more, though. 

  1. Innovation

In the recent resurgence of stop motion animations, strange and amazing props and procedures have been used to create these videos. Filmmakers have gone from generating stop motion with a large sheet of drawings, to constructing them with plasticine figures that need to be manually manipulated millimeters at a time, and to more esoteric props such as foodstuffs, domestic objects, and creatures.

Using this technique, you can animate any object, even one that isn't capable of moving by itself. A stop-motion movie may be made with anything, thus the options are practically limitless.

  1. Animated Tutorials

A wide range of material genres, from educational films to comedic commercials, is now being explored with stop motion animation.

When it comes to creating marketing and instructional videos, stop motion animations is a popular choice due to their adaptability. An individual video can be created. 

Although the film is about five minutes long, viewers are likely to stick with it because of its originality.  The sophisticated tactics employed captivate the audience. Once you start viewing this stop motion video, it's impossible to put it down till the finish.

  1. Improve the perception of your brand

It's easy to remember simple but innovative animations like these. These movies can assist a company's image and later recall be more positive. Stop motion video can provoke thought and awe in viewers, prompting them to spread the creative message to their social networks and professional contacts.

It is becoming increasingly common for organizations of all kinds to include stop-motion animations in their advertisements. 

  1. In education 

Stop-motion films can have a positive impact on both education and business. Employees, customers, and students all benefit from using them to learn difficult concepts and methods more enjoyably. Stop motion filmmaking can liven up any subject matter, and pupils are more likely to retain what they've learned when it's done this way.

Some subjects can be studied more effectively in this way as well. Using stop motion films, for instance, learners can see the entire course of an experiment involving a slow-occurring reaction in a short amount of time.

Learners are given a stop motion assignment to work on as a group project in the classroom. Fast stop motion animation production requires a lot of teamwork, which improves interpersonal skills. Some learners would work on the models, while others might work on the backdrops and voiceovers, while yet others might concentrate on filming the scenes and directing the actors.

  1. Engage Customers and Employees

The usage of stop motion movies can be utilized to explain product uses rapidly, even though the application of the device and the output may take a while. You can speed up the timeline as much as you want in stop motion animations!

For safety and health demonstrations or original sales demonstrations, stop motion instructional films may also be utilized to effectively express complex concepts. Because of the videos' originality, viewers are more likely to pay attention and retain the content.

  1. Music Video

Some incredibly creative music videos have lately been created using stop motion animations, which has recently seen a resurgence in popularity.  Even the human body could be a character in this film.

Stop-motion animations have the potential to be extremely motivating. Sometimes, it's possible to achieve it by presenting things in a novel way, such as by stacking vegetables to appear like moving creatures. The sky's the limit when it comes to what you can dream up.

  1. Reaction-Inducing Video

When it comes to creating a stop motion movie, it doesn't have to be complicated. If you don't have any of these things in your possession, you'll need to get them before you can begin filming. However, if you want to create a professional-level stop motion film, you'll need to enlist the help of an animation company.

As a marketing tool, animated videos may be highly effective when they are created by a professional team. 

  1. Create an Intriguing idea

The story of a motion-capture movie is crucial in attracting the attention of audiences, so it should be carefully planned out before production begins. It should be appropriate for the video's intended audience, brand image, and message. If you need assistance with this, consider working with an animation studio.

Disadvantages

But there are several drawbacks to the overall process of stop motion filmmaking, which are difficult to overcome. The time it takes to create even a min of footage is the most remarkable. The time it takes to get this film might range from a few days to many weeks, depending on the approach used.

Additionally, the amount of time and work that is required to make a stop-motion movie might be enormous. This may necessitate the involvement of a large team. Although this is dependent on the sort of video, stop motion animating is now a fairly broad area of filmmaking, which can require many different talents and approaches.

Conclusion

Using the Raspberry Pi 4, you were able to create a stop-motion movie system. Various stop motion technologies were also covered, along with their advantages and disadvantages. After completing the system's basic functions and integrating additional components of your choice, you're ready to go on to the next phase of programming. Using raspberry pi 4 in the next article, we will build an LED cube.

Running ZeroTier On the Raspberry Pi 4

Our next step in the Raspberry Pi training program is to get zero tiers up and run on a Raspberry Pi 4. How to utilize a Raspberry Pi to measure internet speed and store the results in Grafana or Onedrive was the topic of the last piece. During the project, you will discover how to install ZeroTier on a Raspberry Pi and get it up and running. We will also learn how to set up a firewall to secure our network.

Where To Buy?
No.ComponentsDistributorLink To Buy
1Raspberry Pi 4AmazonBuy Now

Components

  • Raspberry pi 4

  • Power supply

  • Ethernet or wifi

What is zeroTier

ZeroTier is a software that provides a streamlined web-based interface for constructing virtual networks connecting various gadgets. Somewhat akin to configuring a virtual private network on a Raspberry Pi, these networks exist only in cyberspace. The process of provisioning, however, is much easier, especially when dealing with several devices.

Using a decentralized system, all of your gadgets will connect. The end-to-end encryption used to protect these connections helps ensure that your data remains private at all times.

ZeroTier can be used on various platforms, from computers to mobile phones. Its cross-platform compatibility with Unix, Microsoft, and macintosh means you can set up a virtual connection without worrying about whether or not your hardware will be able to connect to it.

The ZeroTier business model is "freemium." Using our free plan, you can connect up to 50 approved devices to the virtual network.

Retrieving your ZeroTier Network ID

You need to create an account on the ZeroTier website before you can use the program on your Raspberry Pi. This is because virtual network administration is performed through their website.

You may manage your entire virtual network from one central web-based console, including assigning permanent IP addresses to individual devices.

Registration on the ZeroTier hub website is required before a network ID can be generated. Access your virtual networks with this web-based interface. Go to ZeroTier Central on whichever browser you like. When you go to the site, look for the "Register" button so you can start the account creation process.

Sign up for ZeroTier by providing the necessary information on the supplied registration form. After you've finished filling out the form, hit the "Register" button at the bottom.

Your account won't be active until you confirm your email address after signing up for it. 

The following window will appear once you've created an account and logged into the web interface. Hit the "Create A Network" button in the screen's center to get started.

When you initially visit ZeroTier and click the button, your first network will automatically generate. The network identification number is listed here. We require this identifier to link your Raspberry Pi to the ZeroTier virtual network. Keep this ID in a safe place; we'll need it soon. Select the entry to change the network's settings, such as the name.

Listed below are the default configuration options for your ZeroTier system. You can modify the network's name, provide a description, and adjust its security level using these options. In addition to the IP subnet, ZeroTier gives you control over many other virtual network features.

Installing ZeroTier to the Raspberry Pi

We can move on now that you've joined ZeroTier and have your network ID. In this part, you'll learn how to download and install ZeroTier on your pi device.

First, let's check that the software on your pi Device is up to date.

To be up-to-date, we need to run the following two instructions for the item list and all installed modules.

sudo apt update

sudo apt upgrade

After adding the GPG key, we can install ZeroTier via their installation repository on our pi Device. With this key, we can ensure that the tools we're installing are directly from ZeroTier and don't include any malicious code. To obtain the GPG key via their repo, type the following code and store the contents of the "de-armored" file in the "/usr/share/keyrings/" folder.

curl https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/doc/contact%40zerotier.com.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/zerotierone-archive-keyring.gpg >/dev/null

Now that the GPG key has been inserted, a source list containing the ZeroTier repository must be compiled. First, we need to create a shell variable named "RELEASE" and assign it the operating system's internal codename. To construct the proper URLs for the ZeroTier repo in the subsequent steps, we will execute the following command.

RELEASE=$(lsb_release -cs)

Once we have the shell variable configured, we can utilize it to construct the relevant ZeroTier repo Urls for the Operating system. We finally save this string in the "/etc/apt/sources.list.d/" folder under the name "zerotier.list."

echo "deb [signed-by=/usr/share/keyrings/zerotierone-archive-keyring.gpg] http://download.zerotier.com/debian/$RELEASE $RELEASE main" | sudo tee /etc/apt/sources.list.d/zerotier.list

The next time you refresh the Raspberry Pi's packages lists, it will pull ZeroTier directly from this location.

Since we have modified the Rpi's source code, we must revise the list of installed packages. Using the command line, you could change your system's component list.

sudo apt update

After updating, we can use the command beforehand to download the ZeroTier package onto our RPi.

sudo apt install -y zerotier-one

ZeroTier can be set up to automatically launch on system startup as part of the setup procedure.

Running ZeroTier on Raspberry Pi 4

Having finished the ZeroTier installation on our RPi, we can now link to the networking we created in the introduction. First, make sure you get the network's identifier handy.

To connect the RPi to the network, we must use the ZeroTier Command line. You can utilize the following code to accomplish this. As a first step, swap out "[NETWORKID]" for the ID you gathered previously in this tutorial.

sudo zerotier-cli join [NETWORKID]

So after this message, your RPi should've just joined the ZeroTier channel.

Joining the ZeroTier channel doesn't make your machine an official part of the network until you verify it. To accomplish this, you must go back to a ZeroTier Main dashboard and change your network configuration. You can also access the site via the following Link, where you should substitute your network Address for "[NETWORKID].

https://my.zerotier.com/network/[NETWORKID]

The "Members" portion is located toward the bottom of the managerial section for the ZeroTier system on the RPi.

  1. You'll need to select the "Auth" box to tick here after identifying the machine you added. As a result, your RPi can communicate with other gadgets on the same network.

  2. A machine through your ZeroTier channel can be located using the information in the "Address" column. The "sudo zerotier-cli status" prompt will cause the RPi to display this data.

  3. The Name/Description field can be used to assign a memorable label to this innovative gadget for future reference.

  4. Lastly, take a peek at the "Managed IPs" section.

If an IP address has been assigned to the gadget, it will appear in this column. These IP addresses will allow you to gain access to that machine. This column can specify which device will receive the IP address. If you're trying to get an Internet address for a newly approved source, be patient; it could take a few minutes.

As soon as your RPi executes the command below, you will know it is linked to the ZeroTier channel. Using this prompt, you can see a complete list of the tracks to which the ZeroTier configuration is currently connected, along with information about the health of those connections.

sudo zerotier-cli listnetworks

Whenever your RPi successfully connects to the ZeroTier networks, you must see something similar to what is shown below. The last number is Pi's Internet protocol address within the VPN connection.

Connecting your RPi to your ZeroTier network must now be complete.

Connecting to other gadgets on the VPN connection is now possible. Having the device's Internet protocol is all that's required. The ZeroTier management console is the quickest way to learn which IP addresses are assigned to particular gadgets. 

Protecting your Raspberry Pi with UFW

Syncthing is a piece of software developed specifically for real-time file synchronization between several devices like the RPi. TLS is used to synchronize data among nodes. Thanks to encryption, the data will be safe and private in transit, making it more challenging to steal. Every node must prove its identity with a rock-solid cryptographic certificate. A connection will be denied to a node if it does not present a valid certificate, preventing unauthorized access. Files on your Microsoft, Macintosh, or Unix computer can be synchronized with your Raspberry Pi via Syncthing. Simple Syncthing apps are accessible for all of these platforms.

Setting Up Syncthing on a Raspberry Pi

Here you can find detailed instructions for setting up your RPi with the Syncthing program. For the program to be installed, we must first add the program's PGP keys and the package repo as possible sources.

We need to upgrade our Rpi before downloading the files synchronization program. We can use the following two commands to keep our RPi up to date.

sudo apt update

sudo apt full-upgrade

Following this, check that the apt-transport-HTTP package has been successfully installed. When using the installer, you can now access sources that utilize the secure Secure protocols, thanks to this package's inclusion. It's not possible to do this by default. This is included by default in most modern operating systems, but it may be missing from lightweight distributions like Raspberry Pi OS Lite. Executing the line below will install the necessary package.

sudo apt install apt-transport-HTTPS

Finally, the Syncthing credentials may be added to our keyrings folder. The purpose of these keys is to verify the authenticity and integrity of the packages we install before trusting them. To obtain the credentials, execute the command that follows on the RPi.

curl -s https://syncthing.net/release-key.txt | gpg --dearmor | sudo tee /usr/share/keyrings/syncthing-archive-keyring.gpg >/dev/null

Since the key has been included, the repo itself may be included. The RPi project will use the Syncthing program, namely the stable release. Use the following command to include the repo in the list of sources.

echo "deb [signed-by=/usr/share/keyrings/syncthing-archive-keyring.gpg] https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list

We have to refresh the installation list before installing Syncthing from the repo. We must revise the list for the package manager to use our different sources. To update your RPI, type the following command into your device's terminal.

sudo apt update

Let's finish setting up our RPi by installing the Syncthing app. Now that the package repository has been added, the program can be installed with a single command.

sudo apt install syncthing

Permitting Third-Party Access to the Syncthing Graphical Interface

The Syncthing web application will only be accessible while close to the device. Those using a Raspberry Pi without a monitor or keyboard would have a very frustrating time if this were the case, but we can change the setup to allow external access.

The first order of business is to discover the RPi's actual local network address. Before proceeding, please ensure that your Rpi has been assigned a permanent IP address. This command lets you find your Pi's local IP address.

hostname –I

To move on, a single iteration of Syncthing must be run to create initial configuration files. The RPI user will be used solely in this tutorial to launch Syncthing.

Syncthing

Press CTRL + C to exit the program after the first launch.

The necessary configurations for Syncthing will be generated after the first execution. The Syncthing program must be launched in the context of the pi user for this configuration file to take effect. With nano editor, start editing the necessary configuration file with the line below.

nano ~/.config/syncthing/config.xml

Locate the following code in this script with the searching key CTRL + W to quickly locate this sentence.

127.0.0.1:8384

This line needs to have the local Internet protocol of our Pi substituted for the default local Internet address (127.0.0.1). For instance, with our Pi's IP address, this code would become something like this.

192.168.0.193:8384

We are limiting our access to people in the same local area network by use of the local Internet address. Alternatively, you can use the internet address "0.0.0.0" to grant access to every IP. Following the successful IP address change, save changes to the script.

Installing Syncthing on a Raspberry Pi as a Web Service

One final step is necessary now that the Syncthing us may be accessed from devices other than the RPi. This responsibility includes developing and launching a system for the program. The Service will enable Syncthing to launch automatically at system boot and be halted and started quickly.

Once again, we'll use nano to make the necessary changes to the Service's configuration file. The Syncthing authorized GitHub is the source for the application we will be developing. To start adding content to the file in "/lib/systemd/system," run the following command.

sudo nano /lib/systemd/system/syncthing.service

copy lines below and paste them to this file.

[Unit]

Description=Syncthing - Open Source Continuous File Synchronization

Documentation=man:syncthing(1)

After=network.target

[Service]

User=pi

ExecStart=/usr/bin/syncthing -no-browser -no-restart -logflags=0

Restart=on-failure

RestartSec=5

SuccessExitStatus=3 4

RestartForceExitStatus=3 4

# Hardening

ProtectSystem=full

PrivateTmp=true

SystemCallArchitectures=native

MemoryDenyWriteExecute=true

NoNewPrivileges=true

[Install]

WantedBy=multi-user.target

Those lines specify how our Rpi's OS must deal with Syncthing. When you're done adding lines, save the file. We could now set up our Service to automatically launch at system startup. Enter this command and hit enter.

Sudo systemctl enable syncthing

Let's run the Service to use the Syncthing internet UI. Once again, the systemctl tool will need to be used to kick off the Service.

sudo systemctl start syncthing

The Syncthing program on the RPi should be checked to ensure it has begun. Using the below program, we can make sure of that.

sudo systemctl status syncthing

The notification should read as follows if the Service was successfully started and is now active.

Web-based Syncthing management through RPi

If everything goes smoothly, you should utilize the Syncthing program on the RPi. Now that the hardware has been added, we can move on to configure the program and synchronize our data. We'll break this up into chunks for easy reading. The web-based user interface makes installing and linking devices a breeze.

Establishing a Link to the Syncthing Online Control Panel

You'll need to launch the web-based interface in your preferred internet browser to begin using it. The Internet address of the RPi is required to use the web-based interface. Using the Url, navigate to the following location in your preferred internet browser.

http://[PIIPADDRESS]:8384

Since the Syncthing program only listens on port 8384, you mustn't remove it from the end of the string.

After creating login details, you will be prompted to sign in before proceeding to the next step.

Safeguarding Syncthing's User Interface

There is no predetermined login information for Syncthing, meaning anyone with access to the UI can change your preferences. Login credentials can be set up to prevent unauthorized users from wreaking havoc.

You will be warned of the potential risks if you have never specified the login details. The "Settings" button on this caution will take us directly to the configuration page.

The configurations page can also be accessed by choosing the "Actions" menu box in the upper right corner and then choosing "Configurations" if this notice isn't shown.

Navigate to the "GUI" tab on the popup settings page. Access this window's account by selecting "GUI" from the window's header.

Login credentials are required to access this page. Passwords should be firm and difficult to guess. Use a mix of alphabetic characters, numeric digits, and special characters. After inputting each, hit the "Save" option to keep your modifications.

After resetting your password, this website will log you out. You'll need to sign in with your new credentials each time you access Syncthing's graphical interface.

How to Get the Syncthing ID of a Device

For Syncthing to function, it must create a random identifier for each connected device. Adding the other device's ID to your own is necessary for sharing information between devices. The RPi Syncthing installation's unique identifier can be located via the web interface.

To return to the main page of the web interface, select "Actions" from the toggle menu in the top right. Select "Show ID" from the selection menu to open the desired dialogue box.

The identification string and corresponding QR code are displayed below. The ideal identifier length is between 50 and 56 characters and may incorporate digits, letters, and hyphens. System-wise, the hyphens are disregarded, but they improve readability. If you want to connect your Raspberry Pi to additional devices, you'll need to give each of them the unique ID assigned to your Pi. You must also include their identification number. Syncthing's mechanism for linking many gadgets to a single pool requires the ID.

Incorporating a New Device into Your Raspberry Pi's Syncthing Network

We've covered how to get your gadget id Number, so now we'll cover adding a new one. Keep in mind that the identifier for your RPi must be entered into whatever gadget you are installing. If not, communication between the devices will be impossible.

The "Add Remote Device" button may be in the lower right corner of the Syncthing UI. When we click this option, we'll be taken to a dialogue where we can add a gadget to our Syncthing collection.

You can add any unique Id to your pool from this dialogue box. Enter the identifier for the gadget you wish to link to the top textbox. Despite its length, the ID is easily copied and pasted. After that, you can connect the device by clicking the "Save" option.

Incorporating each other's Syncthing server Identifiers should result in a successful connection.

Replicating a Directory Across All of Your Devices

Now that we have a device linked to the RPi Syncthing, you can test directory sharing. In this particular chunk, the default directory will suffice. Here, we keep our sync files in a folder called "/home/pi/sync" on our RPi.

Select the "Edit" button next to a directory to change its share settings. We can access the folder's sharing settings dialog by clicking this option and making the necessary changes.

We must navigate to the Share tab under the file settings dialogue. Select the "Share" tab to switch to this view.

The dialog box lets you pick which gadgets will access the shared folder. As soon as you've decided which devices you wish to sync with, choose the "Save" option.

When syncing with the other gadget, keep in mind that you'll need to accept the shared folder on that gadget before synchronizing can begin.

Your directory should have started syncing immediately. When the syncing process is complete, the guide and the gadget should be labeled "Up to current."

Conclusion

Having ZeroTier Syncthing installed on your RPi and linked to a VPN, you may now sync data across machines. If you're looking for a basic virtual network solution, ZeroTier is it. And the best part is that it offers an ideally enough free plan for most people's fundamental needs. Additionally, Syncthing is a user-friendly software that enables you to synchronize folders across several gadgets. The program is among the best methods for allowing many computers to maintain directory consistency in real time. No longer will you have to trust a remote service like Cloud Servers to keep your data safe.

Internet Speed Monitor Using Raspberry Pi 4

Following up on our Raspberry Pi programming course is the next lesson. In the previous post, we learned how to construct an FM radio using a Raspberry Pi. Analog FM broadcasting's circuit construction was also studied in detail. How to use a Raspberry Pi as an internet speed meter and save the data in Grafana or Google Drive is the subject of this article.

You can use this article if you want to keep track of how your downloads, uploads, and ping speeds change over time, and it's easy to use. In addition, you can use this to determine when your internet is at its busiest or if your internet speed has deteriorated. We'll demonstrate how to use Ookla's Internet speed test command-line interface in conjunction with Python code to create an internet speed meter.

The connection speed monitor will employ the Internet speed Command line interface to keep tabs on your connectivity.

Components

  • Raspberry pi 4

  • Micro SD card

  • USB drive

  • Ethernet cable or Wi-Fi

Installing the Speed test CLI

The first step in configuring the RPi to monitoring system the Internet's performance is to ensure the Raspberry is updated. There is an easy way for this using the command line:

sudo apt-get update

sudo apt-get upgrade

To add a repo for the Internet speed Command line software, we have to download a few additional packages. apt-transport-https, dirmngr, & gnupg1 may all be installed on your RPi by running the commands listed below.

sudo apt install apt-transport-https gnupg1 dirmngr

The apt software may now use the HTTPS secure protocols thanks to the apt-transport-HTTPS module. Apt will fail to connect to Ookla's software repository if it doesn't have it. Our Speedtest.net services and your RPi must communicate securely, therefore we'll also set up gnupg1.

Lastly, the dirmngr software is installed. This software is used to add the package repositories to the Rpi's source list. Now that we've installed the necessary tools, we can import the GPG keys for Ookla's Performance test repository into our keychain and start running tests. The performance test CLI interface cannot be downloaded to our RPi without this passcode.

curl -L https://packagecloud.io/ookla/speedtest-cli/gpgkey | gpg --dearmor | sudo tee /usr/share/keyrings/speedtestcli-archive-keyring.gpg >/dev/null

The Ookla repo must be added to our list of sources next. The Performance test CLI cannot be installed on our RPi without the repo being added. The command to add this repo is as follows.

echo "deb [signed-by=/usr/share/keyrings/speedtestcli-archive-keyring.gpg] https://packagecloud.io/ookla/speedtest-cli/debian/ $(lsb_release -cs) main" | sudo tee  /etc/apt/sources.list.d/speedtest.list

You'll see that "$(LSB release -cs)" is used in the command. Input the title of the RPi Operating system release using this string of text in the prompt. We have to upgrade our packages list because we have a new module repository. Simply use the following command to update the list of installed packages.

sudo apt update

Our RPi is now equipped with the official Ookla Connection speed CLI. Installing the software on your device is as simple as running the command below.

sudo apt install speed test

We may now run a speed test on your Raspberry Pi to ensure that we have successfully installed the program. To begin the speed test, enter the following command into your terminal.

Speedtest

There are a few terms of service you must agree to while using the speed test app on your Raspberry Pi. Simply hit "YES" accompanied by the Return key to go past this warning.

Writing our Speed Test Python Script

On our RPi, we can now begin writing our Program code that will actively check the speed of our downloads and uploads. The command prompt will get us started on writing our Program code to check the connection speed on the RPi.

cd ~

nano speedtest.py

Type the code below in this file. We'll walk you through each component of the program, so you can get a sense of how it all works.

import os

import re

import subprocess

import time

This script will use all of the packages listed in these four lines. We'll discuss exactly each of the modules that will be put to use in the following paragraphs.

Import os: 

The script uses the operating system package to interface with the os. This package will be used to see if a file already exists as part of this program.

Import re: 

This repackage provides a library for managing pattern searching so that we may simply perform regular expressions. The Speed test command line provides us with all the information we need to find our desired values.

Import subprocess: 

To run another python code, this script needs the subprocess package. To use the subprocess module, we will be able to launch the Internet speed Command line software and receive the results.

Import time: 

We make use of the time package to keep track of the dates and times of all Speed test Command line calls. We will be able to keep track of the performance over time thanks to this package.

response = subprocess.Popen('/usr/bin/speedtest --accept-license --accept-gdpr', shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8')

Subprocess is used to initiate a request to the Internet speed command line and instruct it to route the output of the speed test to stdout in this section of code. stdout.read is used to output data (). Finally, we decode('UTF-8') our reply variables to make it usable as a Py object after the call to the Speed test Command line.

ping = re.search('Latency:\s+(.*?)\s', response, re.MULTILINE)

download = re.search('Download:\s+(.*?)\s', response, re.MULTILINE)

upload = re.search('Upload:\s+(.*?)\s', response, re.MULTILINE)

jitter = re.search('\((.*?)\s.+jitter\)\s', response, re.MULTILINE)

Each of these 3 pieces of code accomplishes the same task. Every text fragment has a unique number adjacent to it, which they can deduce by running a mathematical equation on it using the re library. A ping lookup for "Latency: 47.943 ms" returns "Latency: 47.943 ms," with only the value between the characters.

ping = ping.group(1)

download = download.group(1)

upload = upload.group(1)

jitter = jitter.group(1)

To retrieve the right numbers, we must utilize the ".group()" function. The CSV file will be able to contain the results of the Speed test Command line software output, thanks to this method.

try:

    f = open('/home/pi/speedtest/speedtest.csv', 'a+')

    if os.stat('/home/pi/speed test/speedtest.csv').st_size == 0:

            f.write('Date,Time,Ping (ms),Jitter (ms),Download (Mbps),Upload (Mbps)\r\n')

except:

    pass

This is a simple piece of code. The program is contained within a try statement, which ensures that the program will continue to run even if an error occurs. First, we retrieve our speedtest.csv document in the try block.

If indeed the document does not already exist, "a+" in the parameters tells it that we wish to generate it and add any new content to what exists already. After that, we use the operating system package to determine the real size of our speedtest.csv documents. If indeed the file's contents are equal to zero, we can proceed. No action is required on our part if the document does not exist.

f.write('{},{},{},{},{},{}\r\n'.format(time.strftime('%m/%d/%y'), time.strftime('%H:%M'), ping, jitter, download, upload))

There are commas to differentiate each record's information. When formatting a string, we utilize the time strftime() method to include the time and current date. Our pings, downloads, and uploads will follow. Example output.

This is what your program should look like when you are done writing it down.

import os

import re

import subprocess

import time

response = subprocess.Popen('/usr/bin/speedtest --accept-license --accept-gdpr', shell=True, stdout=subprocess.PIPE).stdout.read().decode('UTF-8')

ping = re.search('Latency:\s+(.*?)\s', response, re.MULTILINE)

download = re.search('Download:\s+(.*?)\s', response, re.MULTILINE)

upload = re.search('Upload:\s+(.*?)\s', response, re.MULTILINE)

jitter = re.search('\((.*?)\s.+jitter\)\s', response, re.MULTILINE)

ping = ping.group(1)

download = download.group(1)

upload = upload.group(1)

jitter = jitter.group(1)

try:

    f = open('/home/pi/speedtest/speedtest.csv', 'a+')

    if os.stat('/home/pi/speed test/speedtest.csv').st_size == 0:

            f.write('Date,Time,Ping (ms),Jitter (ms),Download (Mbps),Upload (Mbps)\r\n')

except:

    pass

f.write('{},{},{},{},{},{}\r\n'.format(time.strftime('%m/%d/%y'), time.strftime('%H:%M'), ping, jitter, download, upload))

you can save the script. Once our script is complete, we will create a directory in which to keep the speedtest.csv data. Make this directory by typing the command below.

mkdir ~/speedtest

After we have created the necessary directory, we can execute the program. The command below can be used to run our program and see if it works as expected.

python3 ~/speedtest.py

Open the newly generated speedtest.csv file to see the results of the script's execution.  Let's see whether we can open this document on the RPi with the command below.

nano ~/speedtest/speedtest.csv

You should be able to find anything similar to this in that file. A few rows of records and the column headings.

It is possible to use this guide in two ways. Either utilize Influx Database and Grafana or use google Drive to synchronize the data to Google Account using Influx Database and Grafana.

How to View Your Speedtest Results in Grafana?

We'll teach you to easily plot your performance test data using Grafana throughout this section. To conduct data analytics, load up metrics that make some sense of the immense amount of data, and track our applications with the aid of cool configurable panels, we use Grafana, a free software solution that is free and open source.  In addition to the fact that Grafana is an open-source platform, we may create our plugins to integrate with a variety of data sources.

Technically known as time series analytics, the technology aids in the study, analysis, and monitoring of data across time. By giving relative data, it aids us in tracking user activity, app behavior patterns, error rate, error kind, and contextual circumstances in operation or a pre-production scenario.

Organizations that are concerned about security or other factors do not have to use the vendor cloud because the project can be implemented on-premise. Over the years, this framework has become an industry standard and is used by companies like PayPal, eBay, Intel, and many more. In a moment, I'll go over some real-world examples from the industry.

Grafana Platform & Enterprise are 2 extra services provided by the Grafana developers for companies in addition to the free software core product. What do they do? The remainder of this post will go into greater detail regarding this. In the meantime, how about we take a closer look at the tool's capabilities and architecture flow, starting with an explanation of what a panel is? & How does it all work? '

Grafana Panel: What Is It?

They use sources of data like Graphite and Prometheus as well as Influx database and Elastic Search to populate the panels. Grafana has built-in compatibility for a wide range of data sources, including these.

All of the common charts and graphs that a business needs to analyze data can be found in the panel's visualization options, including heat maps, scatterplots, and more. On the grid of a dashboard, there are several discrete panels. There are a variety of features available on each panel.

What are Grafana's Capabilities?

Let's have a look at the fully accessible panel framework's capabilities. Our application's metrics are handled via an open platform. This data can be analyzed through the use of metrics in a variety of ways.

The panel is well-equipped to generate a sense of complicated data, and it is constantly changing. Geo-mapping, heat maps, scatterplots, and more can be displayed with graphs in a variety of ways. Our business needs can be met by a wide range of data presentation possibilities provided by the software.

As soon as a predetermined event occurs, an alert is set up and triggered. Slack or any other communication tool used by the monitoring team might be alerted to these events. Grafana is pre-installed with support for about a dozen different types of databases. And there is a slew of more, all made possible thanks to plugins.

It can be hosted on-premises or in the cloud. Custom data can be retrieved using built-in Graphite support and expressions such as "add," "filter," "average," "minimum," and "maximum" functions. Graphite is a chemical element. Later, I'll address that. Influx database, Prometheus, Elastic Search, and Cloud Monitoring are also included. Up front, I'll cover it all.

What is the purpose of Grafana Cloud?

A cloud-native, highly accessible, quick, and completely open SaaS metric framework, Grafana Cloud As a result, individuals who don't want to host the solution on their own and prefer to avoid the headache of managing their deployment infrastructure may find this useful. It's a Kubernetes-based service. Prometheus and Graphite back end is supported. This gives us two options: either use Grafana on-premises or both.

Setting up the Influx Database for our Connection Speed Monitoring

Installing Influx Database on your RPi is a prerequisite for this stage of the internet speed monitoring guide. Our connection speed monitoring system sends data to this location, thus we'll be storing it here.

Influx database

Designed by Influx Intelligence, Influx Database is a free and open-source time series system built in Go. Time series data, such as that collected from sensors and IoT devices, may be accessed quickly and reliably with this system because of its focus on high-availability extraction and retention. As a Time Series Database, Influx Database is capable of storing up to several hundred thousand points each second. A SQL-like query language for time series data, the Influx Database was designed expressly for this purpose.

Why use Influx DB?

  • Shorter duration

  • Extensive research and analysis

  • Retention, ingestion, querying, and visualization are now all available through a single application programming interface in Influx Database.

  • Templates that are simple to create and distribute, thanks to the influx of DB templates

First, we'll fire up the Influx Database CLI tool by typing the command below. Using this application, we will be creating an online repository for our data.

There is no need to enter the passcode and username for Influx Database if you haven't set login. Establish a database with the name "internet speed" in it immediately. After typing CREATE DATABASE, the DB name, and pressing enter, the DB is ready to use.

CREATE DATABASE internet speed

Creating a user named "speed monitor" will be the next phase in working with the database. The passcode "pass" should be replaced by a more secure one. Privileges are not a concern at this time, as we shall take care of them in the following stage.

CREATE USER "speed monitor" WITH PASSWORD 'pass' Now assign the new "speed monitor" user all privileges to our "internet speed" database.

GRANT ALL ON "internet speed" to "speed monitor"

To shut off the application, type the command below.

Quit

Installing the Python package required to communicate with the Influx DB is the final step.

sudo apt install python3-influxdb

Saving our Speed Monitor Data to our Influx DB

Create a new Script file to start populating our Influx database now that it has been set up. If you've already read through the previous script, you won't have to go over anything new here.

rm ~/speedtest.py

nano ~/speedtest.py

To get started, we have to include all of the Python packages that we will be using in this file.

import re

import subprocess

from influxdb import InfluxDBClient

operating system and time have been eliminated, as seen. We no longer have to communicate with records, and the Influx database automatically timestamps data, therefore these two libraries are no longer required. After importing the "InfluxDBClient" for our Influx database server, we are ready to use it. The next phase is to launch the Speedtest Command line interface and process the results. Upon completion of this code snippet, we'll have all the information we need.

response = subprocess.Popen('/usr/bin/speedtest --accept-license --accept-gdpr',

                            shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8')

ping = re.search('Latency:\s+(.*?)\s', response, re.MULTILINE)

download = re.search('Download:\s+(.*?)\s', response, re.MULTILINE)

upload = re.search('Upload:\s+(.*?)\s', response, re.MULTILINE)

jitter = re.search('\((.*?)\s.+jitter\)\s', response, re.MULTILINE)

ping = ping.group(1)

download = download.group(1)

upload = upload.group(1)

jitter = jitter.group(1)

Now everything gets a little more complicated. This data must be converted to a Py dictionary for us to use it. Because the library wants the information to be presented in a JSON-like form, this is an explanation.

speed_data = [

    {

        "measurement" : "internet_speed",

        "tags" : {

            "host": "Raspberrytheengineeringprojects"

        },

        "fields" : {

            "download": float(download),

            "upload": float(upload),

            "ping": float(ping),

            "jitter": float(jitter)

        }

    }

]

In this section, we established our dictionaries by the Influx database data model. "internet speed" is the title we assigned the metric. The tag "host" was also added so that if we were to manage numerous devices within the same DB, we could segregate them. After that, we enter the data we obtained in the preceding line of code, including the download speed, upload speed, and pings.

To make them into numbers, we use the float () method to turn our download, uploads, and pings parameters into strings. Grafana will read these as characters if we don't utilize the float () method. Now that we have all the information we need, we can begin using Influx Database. It is necessary to create an InfluxDBClient object and provide the network information.

Only the hostname, port number, user id, passcode, and DB name are passed to this method. You can refer to the official Python manual for Influx Database if you wish to know what information can be set.

client = InfluxDBClient('localhost', 8086, 'speedmonitor', 'pass', 'internetspeed')

"localhost" should be replaced with the Internet address of your Influx database server if it is hosted elsewhere. Change "pass" to the passcode you created earlier in this article. To send data to our Influx database server, we need to add a block of code like the one below to our existing codebase.

client.write_points(speed_data)

To send data to Influx Database, we only need to do that. Assuming you've entered every bit of code in the document, this should look something like this.

import re

import subprocess

from influxdb import InfluxDBClient

response = subprocess.Popen('/usr/bin/speedtest --accept-license --accept-gdpr',

                            shell=True, stdout=subprocess.PIPE).stdout.read().decode('UTF-8')

ping = re.search('Latency:\s+(.*?)\s', response, re.MULTILINE)

download = re.search('Download:\s+(.*?)\s', response, re.MULTILINE)

upload = re.search('Upload:\s+(.*?)\s', response, re.MULTILINE)

jitter = re.search('\((.*?)\s.+jitter\)\s', response, re.MULTILINE)

ping = ping.group(1)

download = download.group(1)

upload = upload.group(1)

jitter = jitter.group(1)

speed_data = [

    {

        "measurement" : "internet_speed",

        "tags" : {

            "host": "Raspberrytheengineeringprojects"

        },

        "fields" : {

            "download": float(download),

            "upload": float(upload),

            "ping": float(ping),

            "jitter": float(jitter)

        }

    }

]

client = InfluxDBClient('localhost', 8086, 'speed monitor', pass, 'internet speed')

client.write_points(speed_data)

Save the document to your computer.

Using Grafana to see our Connection Speed Stats

The database needs to be displayed in Grafana. All the information will be graphed and shown by using the Grafana application.

Grafana installation on the Raspberry Pi.

It's a fully accessible metric monitoring and data presentation package for people who aren't familiar with it. The purpose of this software is to aid in the visual representation of time-based information. To speed things up, Grafana entrusts most of the heavy lifting to the client, such as generating graphs. Since there are minimal data to analyze, the software can concentrate on giving information that can be used to create graphs.

Many various types of data sources are supported by Grafana, and you can set up warning rules for key metrics to be warned immediately if anything isn't quite right. The famous Influx Database is one of the available Grafana data capture options. Quick and "time series" oriented, Influx Database is a famous system for use with Grafana. Each register contains a timestamp.

Grafana is frequently used to keep tabs on system metrics like the temperatures of the equipment and how much of it is being used. In addition, it can be used to graph data, for example, the weather, across time. Grafana is an excellent tool for instantly presenting data from your Raspberry Pi.

Setting up a Raspberry Pi with Grafana

It's a good idea to double-check that all of the packages on your RPi are updated before beginning the Grafana installation. The 2 techniques listed below can be used to do this. The packages list will be updated, and all installed applications will be upgraded to the most recent versions using these instructions.

sudo apt update

sudo apt upgrade

The Grafana source repo must be added to the RPi before Grafana can be installed. As a prerequisite, we must add an APT password. Using the APT password, you can confirm that the modules you're installing originated from the Grafana packages service and are properly signed. The instruction to include the Grafana APT password to your RPi's keychain is as follows.

curl https://packages.grafana.com/gpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/grafana-archive-keyrings.gpg >/dev/null

Once we've uploaded the password to our Raspberry, we're good to go with the Grafana repo as a resource for our software. Include this repo to the source list by running the command below on your RPi.

echo "deb [signed-by=/usr/share/keyrings/grafana-archive-keyrings.gpg] https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list

The RPi will automatically check the Grafana repo for new packages whenever you launch and upgrade them. An update is necessary because we've added new packages to our list. When using apt to perform an update, the most up-to-date package list is obtained from all available sources. To accomplish this, run the command below in the console of your Raspberry.

sudo apt update

Please keep in mind that Grafana can be installed on your RPI. Run the command below to install the newest release of Grafana on your computer.

sudo apt install grafana

Getting Grafana to start automatically at startup is the next step we need to take. Grafana includes a systemd service file, which is a godsend for those of us using it on Linux systems. All we have to do is execute the command below to make Grafana start automatically at system startup.

sudo systemctl enable grafana-server

The "grafana-server.service" services record will be enabled by this instruction to a network's service management. The Grafana server's service management will utilize this file as a reference guide. In the console of the Raspberry Pi, enter the following command to begin using Grafana's webserver.

sudo systemctl start grafana-server

Setting up the Grafana dashboard on your Pi 4 

Now that we've installed Grafana on your Pi 4, we can use its web interface to monitor your data. If you have a Raspberry Pi, the first thing we'll need to do is get its Internet address. Grafana on your local area network can be accessed remotely via this Internet protocol. The IP address of your Raspberry Pi may be found by typing the following code.

hostname –I

Static IPs are a good idea if you frequently need to connect to your Raspberry Pi. Make sure you have your Internet Protocol (IP) address available before visiting this URL. A web application for the Grafana dashboard can be found on line 3000 of the Rasp Internet address. "IPADDRESS>" should be replaced with your Internet address from earlier.

When you initially open Grafana, you'll get a login page. When you initially installed Grafana on the RPi, you were given the option of logging in with the default administrator account. The username and passcode are "admin" and "admin," respectively, for this account (1.). However, even though the passphrase is incredibly insecure, we'll be able to alter it right after this one. Grafana's "Login" tab can be clicked once the userid and passcode have been entered.

Grafana's online UI will prompt you to update the user's passcode upon the first login. You can skip this step if you like, but we don't encourage it. You must change the passcode as soon as possible due to its extreme insecurity. After entering a new pass, select the "Save" option to continue to the Grafana homepage.

The below screen will appear after you have signed in and updated the passcode. Now that you've reached this screen, you're ready to begin configuring the Grafana interface on your RPI.

Now that Grafana is installed on the RPi, you can access the online interface. To continue with this article, access the Grafana graphical interface once it has been installed. If you're not sure where it is, type http://localhost:3000 into your browser's address bar.

IPADDRESS:3000

A new information source must be added to Grafana's web app. ' The "Data Sources" menu selection can be accessed by clicking on the wheel on the left (1.). 

To continue, click "Add source of data" in the following menu.

Navigate to the "Add data" webpage then click the "Select" tab when you discover Influx Database.

We must now fill in the data regarding the Influx Database configuration. Enter the Influx Database Hyperlink first (1.). In our scenario, the RPi which we are using to execute the program is where this is located. Use the following Link if you're following suit.

http://localhost:8086

The credentials for our DB must then be entered (2.). The Db must be set to "internetspeed" if you closely followed our instructions. Last but not least, the passcode must be the one we mentioned; if you utilize our examples, it is "theengineeringprojects". The Username should be "speedmonitor,". After you've entered all the necessary data, select the "Save & Test" tab (3.)

The following step is to build a panel to show this information. Select the "Dashboard" option by hovering your cursor over the add (+) button in the navigation bar.

Select "Add Query" from the menu that appears over "New Panel."

Graph reading from our DB can now be configured. The metric name we want to use is "internetspeed," so click "choose measurement" (1.) after that (2.)

We must now establish the data collected that we want to examine after we have the metric set. The "value" wording adjacent to the field can be clicked (1.). Click "downloads," "uploads," or "ping" from the drop-down menu. We'll begin with "download" in this tutorial (2.).

After that, we'd like Grafana to consider each outcome as a separate entity. In the picking row, click on the add (+) tab to accomplish this (1.). In the pop-up, select "distinct" from "Aggregations" (2.).

If you want to comprehend this data better, it's time to give it a new name. Assign an alias to each field that accurately reflects what it contains (1.). "Download Pace," for example, is the name of the speed at which data is downloaded. It's time to repeat these procedures 7–9 till all three variables (downloads, uploads, and pings) have been included in the graph.

Add every field by clicking the wrench and wheel symbols on the navigation bar, and then click "Save".

Using the "Title" box (1.) allows you to enter the name of the graph's title. As a title, "Network Speed " was utilized in this case. The panel can be saved by tapping on the save option (2.)

Give your panel a name (1.). You have a complete creative license with this one. Our network bandwidth monitor was given the name "Network Speed Monitor" by us. Afterward, you may save this panel by selecting "Save" (2.).

Your statistics should now be visible on a visually appealing graph, assuming all went according to plan.

Grafana used to automate your Speed Monitor scripts

Making your program run on a regular schedule is as simple as automation.  The crontab is the simplest approach to schedule your script to execute regularly. On your RPi, you can change the crontab by typing the command below.

crontab –e

When asked which editor should use, we suggest nano because it's the simplest to learn and the most intuitive. The following cronjob should be added at the bottom of this file. Cronjobs are scheduled to run each half an hour by default. We advise using our Crontab generator if you'd like to experiment with alternative timings.

Exactly what does it mean to have a "Cron Job?"

Jobs are scheduled using Cron, which is built into Unix-like systems like Linux and its numerous variants. It is a time-based mechanism. Using the cron is a common approach to run instructions or bash scripts regularly. "Cron Jobs" refers to tasks that are scheduled using the "cron" utility. While using Unix-based systems like Raspbian, you'll quickly become dependent on cron jobs.

Using Google Drive to store your Internet Speed Readings

gDrive Installation and Compilation

It's easy to use gDrive, a cli program, to transmit to Google Account. Once you've got it established on the smartphone, it's a breeze to use. This instruction will explain to you how to use your personal Google accounts to develop the gDrive program on the RPi. The same procedures can be used to create gDrive for any os, even if this instruction concentrates on the RPi. 

Getting the Go Compiler installed

The Go engine must be installed on our device before we can assemble the gDrive program. Download the appropriate drivers from the official website whether you're working on a PC or Mac.

If you're working with a Linux distribution like Raspbian, the process becomes a little more complicated. Using a Linux terminal, type one of these commands. 

The Raspberry Pi can be used with this.

wget https://dl.google.com/go/go1.13.7.linux-armv6l.tar.gz -O go.tar.gz

a 64-bit version of Linux

wget https://dl.google.com/go/go1.13.7.linux-amd64.tar.gz -O go.tar.gz

After downloading the Go libraries, we must now unpack them to the root directory. 

sudo tar -C /usr/local -xzf go.tar.gz

Next, we'll see whether we can get the console to talk to Go. If we alter the shell aliases script, we can accomplish this goal. Shell will run automatically the script and pull in our updated path names.

nano ~/.bashrc

The following lines should be added to the end of this file. With these lines, we may execute the compiler instantly from the cli, without having to specify the directory to the engine.

export GOPATH=$HOME/go

export PATH=/usr/local/go/bin:$PATH:$GOPATH/bin

Now you may save the script.

source ~/.bashrc

Retrieving your Google Drive API Credentials

We require your Google Cloud Apis details before we can start with the gDrive program compilation. " Your project's name can be found on this webpage (1.). "gDrive-theengineeringprojects" shall be the name of our example.

For our project, we'll need to activate the Google Cloud Application programming interface first. In the Cloud Services panel, we can access the Google Cloud Application programming interface page. Select ENABLE from the drop-down menu.

When you get to the next screen, select the "Credentials" tab from the navigation bar (1.). "Config Permission SCREEN" needs to be clicked next to finish the process (2.).

You'll be prompted to select a "User Type" at this point. "External" is the only option available if you are not running a business (1.). Once you've made your choice, click "Start" to begin building your app (2.).

You'll need to give your app a name on this screen (1.). In this case, we'll use the name "gDrive theengineeringprojects" for the program.

To save the document, you simply need to type in a title for your program.

Return to the "Credentials" page, and then we'll be done.

The navigation bar makes it simple to return to the previous page.

Add credentials by clicking on this tab: "Create Credentials" (1.). When the pop-up displays, select the "OAuth user ID" option (2.).

Selecting an app type is what we need to do next. We chose "Other" since none of the other options were appropriate for the API's intended use. Once we've done that, we'll need to give this program a name. We'll just call it "gDrive theengineeringprojects" for the sake of simplicity. Once all of the information has been input, click the "Create" tab to begin the process.

You should now have both the client id and client secret.

In this article, you'll need each of these numbers at some point.

Setting up your API Keys for gDrive

We'll need to use git to download gDrive's source code before we can compile it. Before we can proceed, we need to install the git client on our computer. To install Git on a Debian-based operating system like Linux or Raspbian, you may either go to the main Git webpage or use the procedures below.

sudo apt update

Just type the command below and we'll be done in no time.

sudo apt install git

now clone git

git clone https://github.com/gdrive-org/gdrive

The next step is to update the program to reflect the new client password and session id. Make a copy of the "handlers meta.go" file in the cloned subdirectory and edit it with nano.

cd gdrive

nano handlers_drive.go

Change the collected details in the following statement of this file. Both your user id and password should be in your possession.

const ClientId = "367116221053-7n0vf5akeru7on6o2fjinrecpdoe99eg.apps.googleusercontent.com"

Substitute your login Credential here.

const ClientId = "YOURCLIENTID"

const ClientSecret = "1qsNodXNaWq1mQuBjUjmvhoO"

You can use your user password instead

const ClientSecret = "YOURCLIENTSECRET"

Save all the changes. Now it's time to execute the following code to get the additional modules needed to compile our updated version of gDrive using the Go engine.

go get github.com/prasmussen/gdrive

To get gDrive working on our device, simply enter the command shown below into your terminal.

go build -ldflags '-w -s'

It's time to get this thing working on the command line, so let's get started! We need to relocate the file to the root directory to use the gdrive inside the cli. To relocate the executables, type the command below.

sudo mv gdrive /usr/local/bin/gdrive

The final step is to provide the gdrive file with the ability to run.

sudo chmod a+x /usr/local/bin/gdrive

It's time to play around with gDrive

Now that your Google account is linked to the app, we can test the program gDrive. The gdrive instruction and the "list" parameter are required to get things started.

gdrive list

Following gDrive's list statement, you will be informed that authorization is necessary. There needs to be a Hyperlink at the bottom of the message. Using your Google acc, users must visit this Address and sign in. You'll get a security code if you perform the next few steps on the internet browser. Enter the verification code that you just copied into the terminal.

GDrive has been successfully installed onto your device if a listing of files is displayed. To see the ids for each of your directories, you can use this command. Using the IDs listed above, you can sync a specific folder.  The command below can be used to test syncing a folder. You can replace Folder> with the path to your synchronized folders.

The identification of a directory that you obtained with the grdive listing commands must be substituted for GOOGLEFOLDERID>.

Uploading Speed Test Data to Google Drive

Now that gDrive is installed on the RPi, we're ready to collect some speed test results. Using gDrive, establish a new directory on the Google drive account for our speedtest.csv record. This will be our starting point. This next terminal command will allow us to accomplish this.

gdrive mkdir speedtest

A notification stating that the subdirectory has been established will be displayed as a result of running this command. This mail will also provide you with your identification number. Write this Identification down someplace safe; we'll need it in a few stages. We may now utilize the subdirectories Identification to add a file to it, as the directory has been created. The speedtest.csv record will be used in this experiment. Be careful to substitute YOUR FOLDER ID with the identification you received in the previous phase before running the command below.

gdrive sync upload ~/speedtest YOUR_FOLDER_ID

The command prompt should display something like the one below during the first sync. Messages such as this one inform you that document has been successfully transferred to your Onedrive.

In the meantime, we'll be able to automate the entire process such that we submit the speedtest.csv whenever we make changes to it.

Using Google Drive to automate the Rpi's connection speed monitoring

Automating your Raspberry Connection Speed Monitoring is the following main task related to it. We'll be building a shell script to automate the process. Crontab will use this script to run it regularly. Use the following Unix commands on the RPi to get started developing the shell script.

nano ~/speedtest.sh

The following lines are what we'd like to include in this document. Your Google storage subdirectories unique ID must be replaced by YOUR FOLDER Identification.

#!/bin/bash

python3 /home/pi/speedtest.py

/usr/local/bin/gdrive sync upload  /home/pi/speedtest YOUR_FOLDER_ID

Save the script. Our shell script needs to be granted permission to run before we can set up a crontab in which to run it. By entering the command below into the prompt, we can accomplish our goal!

sudo chmod +x /home/pi/speedtest.sh

We're now ready to set up the crontab now that everything is finished. Start by executing the command below on the RPi to begin modifying the crontab. When prompted, choose Nano as your editor of choice.

crontab –e

At the end of the document, paste the following code. This command tells crontab to execute our shell scripts once every hour, which it will do. Our Crontab generator can help you come up with new values for the crontab if you'd like.

0 * * * * /home/pi/speedtest.sh

Conclusion

We learned how to set up a pi 4 internet connection test monitoring in this article. We also learned how to set up the internet monitoring system's influx database and grafana application. Now you can experiment with other servers to see if you can enhance the speed test's precision and performance. We're going to use our Raspberry Pi 4 to develop a Wi-Fi gateway in the next tutorial.

Security System with Image Capturing in Raspberry Pi 4

Thank you for joining us for yet another session of this series on Raspberry Pi programming. In the preceding tutorial, we constructed a personal Twitter bot using Tweepy, a Py framework for querying the Twitter application programming interface. We also constructed a Response to robot mentions that would post a response to everybody's tweet mentioning it with a certain keyword. However, in this tutorial, we will implement a security system using a motion sensor with an alarm.

This is what it looks like:

PIR Motion Sensors can be implemented with RPi by understanding how it is connected to a Raspberry Pi. Whenever the motion sensor detects human movement, an alarm is triggered in this project and the LEDs blink. You may create a simple motion-detection alarm using this interface.

Overview

Infrared Motion Detectors or PIR Sensors are Motion Sensors that use Infrared Radiation to detect movement.

Infrared rays are emitted by anything with a temperature higher than absolute zero, be it life or non-living. Humans are unable to see infrared radiation because its wavelength is longer than the wavelength of visible light.

That's why PIR Sensors are designed to pick up on those infrared rays. Due to their wide range of uses, such as motion sensors for security systems and intruder alert devices

"Passive" in motion sensor refers to the fact that it doesn't produce any radiant rays of its own, but rather detects it when other things emit infrared radiation. This is in contrast to active detectors, which perform both the generation of infrared waves and the detection of these waves simultaneously.

An Overview of Motion Detectors

For this project, we used a motion detector that included an infrared sensor, a BISS0001 integrated circuit, and other parts.

The 3 pins on the motion sensor are used for power, data, and ground. There are two potentiometers on the Motion Sensor that may be used to modify both the sensor's sensitivity and the period it remains high on sensing a body movement.

A key role in directing infrared rays onto the sensor is played by the Fresnel lens overlaying the Pyroelectric Sensor. This lens allows the PIR Sensor to detect things at an angle of 1200 degrees. The sensor has an 8-meter detection range, meaning it can pick up on human movement within that distance.

PIR Sensor Adjustments

Two potentiometers are provided for fine-tuning the sensor and output timing, as previously described.

With the aid of a potentiometer, you may modify the sensor's sensitivity. The distance can be changed between 3m and eight meters. To increase the detecting distance, spin the Potentiometer in a clockwise motion and to reduce, rotate it in the opposite direction.

The second potentiometer allows you to choose how long the motion sensor's output remains HIGH. Anywhere from 0.3s to 600s can be used. Turn the POT clockwise to raise the time and the opposite turn to decrease it.

PIR Motion Sensor with Raspberry Pi 4

A Motion Sensor based on RPi and Python language has been the goal of this project since the beginning, as stated in the intro.

I have an Infrared Motion Sensor Component in numerous different projects like Automated Lighting using Raspberry and Various Sensors, Automated Door Opening with Arduino and a motion sensor, and GSM Home Automation Security with Pi.

The key advantage of the Infrared Motion Sensor utilizing RPi over the above-described projects is that RPi can be readily connected to the Web and allows Internet of things implementation of the project.

Circuit Diagram

The following figure illustrates the interfaces concerning the Infrared Motion Detector using RPi.

Components Required

  • Raspberry Pi 4

  • PIR Sensor

  • Speaker

  • Jumper Wires

  • Breadboard

  • Power Supply

  • Computer

Circuit Design

Link the Motion Sensor's Vin and GND connectors to the RPi's 5 volts and GND pins. Use pin11 to attach the Infrared Sensor's DATA Input.

Gnd and pin 3 are where you'll want to connect the led. As soon as the sensor is triggered, these LEDs will come on and go off.

Code

Python is used for the programming portion of the project. The Python program for RPi's infrared Motion Sensor is provided below. Insert the program into a new file called motion.py.

import RPi.GPIO as GPIO

import time

GPIO.setwarnings(False)

GPIO.setmode(GPIO.BOARD)

GPIO.setup(11, GPIO.IN) #Read output from PIR motion sensor

GPIO.setup(3, GPIO.OUT) #LED output pin

while True:

i=GPIO.input(11)

if i==0: #When output from motion sensor is LOW

print("No intruders",i)

GPIO.output(3, 0) #Turn OFF LED

time.sleep(0.1)

elif i==1: #When output from motion sensor is HIGH

print("Intruder detected",i)

GPIO.output(3, 1) #Turn ON LED

time.sleep(0.1)

How it works

The operation of the Infrared Motion Sensor with Raspberry Pi is pretty straightforward. If the Infrared sensor senses some body motion, it sets the Data Input to HIGH.

RPI on identifying a 1 on the associated input gpio, will trigger the alarm.

If the PIR sensor is not working:

When you purchase a new sensor, it doesn't work. The Trim port is in the default setting, so it's not a sensor issue. Sensitivity of the sensor and trigger duration port if you modify these settings. It's going to start working as planned. Make sure the trigger duration port's knob is on the left as a low trigger duration and the sensitivity port is in the middle.

Applications

Infrared Motion Sensor with Raspberry Pi has already been discussed. They include:

  • Automated house lights

  • Motion sensing

  • Intruders notice

  • Automated door open

  • Home security systems

When motion is detected by the PIR sensor on the raspberry pi, we will look into how to record video and transmit it to Whatsapp as an alarm. So that we can tell who's in your room right away thanks to the photo.

Capture an image with the pi camera

Enable the camera by going to the Preferences menu and selecting the Raspberry Pi configuration option.

Activating the camera and saving the image will allow us to identify who or what triggered the alarm.

Python code

import picamera

from time import sleep

camera = picamera.PiCamera()

camera.capture('image.jpg')

When we run our software, the preceding code will take a picture and put it inside the root directory of the script. This image will be used to identify the intruder that has been detected.

Sound alarm

When an alarm system is triggered, there is an alert that must sound. We'll use a loudspeaker instead of a buzzer for our alarm system in this scenario. When the motion sensor is activated, we will play an alarm sound.

Code:

import pygame

pygame.mixer.init()

pygame.mixer.music.load("alarm.mp3")

pygame.mixer.music.play()

while pygame.mixer.music.get_busy() == True:

continue

As a bridge python software for video game design, Pygame is an excellent choice. Additionally, it provides sights, sounds, and visualizations that can improve the game that is being created.

Graphics for video games can be generated using a variety of libraries that deal with visuals and sounds. It streamlines the entire game workflow and makes it easier for newcomers who wish to create games.

Copy the code above and save it to a file named alarm.py then run it in the terminal.

python alarm.py

Send an image to Whatsapp using Twilio

Any internet or mobile app's compatibility with several platforms was a major hurdle to overcome when designing it. It used to be possible to build a link between two pieces of software using Bandwidth or Podium or Telnyx or Zipwhip or similar before Twilio was invented. In recent years, though, Twilio has dominated the competition. Twilio has become the preferred communication API for programmers. Twilio will become clearer to you if you stick around for a time.

What is Twilio

Developers can use Twilio's API to communicate with each other in a modern way.

When it comes to creating the ideal client experience, developers have a wealth of tools at their disposal in the form of Twilio's APIs, which the company describes as "a set of building blocks."

It is possible to utilize Twilio to communicate with customers via text message (SMS), WhatsApp, voice, video, and email. Your software only needs to be integrated with the API.

What does Twilio do?

Twilio is a provider of end-to-end solutions for integrating voice and data communication. Twilio is already used by over a billion developers and some of the world's most well-known businesses. The Twilio Communication application programming interface enables web and mobile app developers to integrate voice, message, and video conferencing capabilities. This makes it easier for app developers to communicate with one another.

The API provided by Twilio makes it simple and accessible to communicate across the web. Mobile and web applications can use this service to make phone calls as well as send text messages and multimedia messages (MMS).

How Does Twilio Work?

You might want to learn more about Twilio and how it works. As a result, Twilio allows enterprises to better understand their customers than any other service. Twilio's primary concept is to acquire clients, get to know them, provide for their needs, and keep them coming back.

Twilio has a worldwide operations center that keeps an eye on carrier networks around the clock to ensure that they are operating at peak efficiency. To keep up with the ever-changing traffic patterns, Twilio's skilled communications engineers are on the job all the time.

They employ real-time feedback from several provider services to make smarter routing decisions based on real-time data on the availability of handsets. The key distinction between Twilio and other application programming interface integration networks is that Twilio's data-centric strategy provides customer engagement service.

Key Areas Of Twilio

Contact Center

Managing a contact center in today's business environment is critical to the success of the company. Businesses can use Twilio to manage their interactions with clients and consumers through a central contact center platform.

Messaging

Before Twilight, sending mass SMS was a difficult task. Now, the Twilio Message application programming interface is widely used to transmit and receive messages, MMS, and OTT communications worldwide. Users can verify whether or not messages have been delivered using the intelligence tracking services.

Videos

For healthcare, virtual classrooms, recruiting, and other uses, Twilio's WebRTC and cloud infrastructure components make it easy for developers to create secure, video, and HD audio applications.

Marketing Campaigns

Twilio's ability to run and manage marketing campaigns is another noteworthy but still-evolving feature. Users can examine performance numbers, run campaigns, and view design concepts.

Voice

As a result of this trend, Twilio has also seen an increase in voice traffic. Any app, website, or service can use Twilio to make phone calls over the PSTN or SIP. It's easy to use Twilio Programmable Voice to make and manage digital calls for any campaign.

Email

The Twilio SendGrip application programming interface eliminates the issue of emails that never make it to their intended recipient's inbox. Customers and clients will receive your emails with Twilio, so you won't have any worries about them not getting them.

User Verification

You'll never have to worry about online scams or fraud again using Twilio's verify feature. It is continuously validated by SMS, Voice, email, and push alerts continuously.

Connectivity

Advancing solutions and services provided by Twilio allow for global connectivity. As a result of this connectedness, your company can grow with ease.

Obtain Twilio credentials

The Twilio WhatsApp sandbox

Developing and testing your app is made simple using Twilio's WhatsApp Sandbox. Your Twilio mobile number must be approved by WhatsApp before you can seek production access.

You'll learn how to connect your phone to the environment in this section. Select Messaging in the Twilio Console and then Take a look at the WhatsApp section by clicking on it. On the webpage, you'll find the information you need to join our sandbox.

The word "join" will be the first character in the code, followed by a two-word phrase chosen at random.

As soon as Twilio receives your message, you should be able to send and receive text messages on your cell phone without any issues.

Please repeat the sandbox application process for each additional mobile device that you wish to use to test the application

Configuration

Set up a new Python project in the following section.

mkdir python-whatsapp-pic

cd python-whatsapp-pic

We'll need a virtual space for this project because we'll be installing several Python packages.

Open a terminal on your RPI machine and type:

python -m venv venv

source venv/bin/activate

(venv) $ pip3 install twilio

When using a PC running Windows, execute these commands from a command line.

python -m venv venv

source venv\bin\activate

(venv) $ pip3 install twilio

Python's Twilio library will be used to deliver messages via Twilio.

Authenticate against Twilio services

To authenticate with the Twilio service, we must safely store a few critical credentials. To use Twilio we need to register for an account at the official Twilio website. Create a new account with your email and password. They will send a confirmation message to your email inbox for you to confirm the registration. Go ahead and confirm it. You will also have to verify your WhatsApp phone number to proceed.

Setting environment variables can be done by entering the code below into your terminal:

ssh auth token

export TWILIO_ACCOUNT_SID="your account sid"

export TWILIO_AUTH_TOKEN= "your auth token"

after we have exported the credentials in our environment, the next step is to activate the WhatsApp sandbox to receive messages. Go to the develop mode, then select messaging and send a Whatsapp message.

You will see a message directing you to deliver a text to your phone and if Whatsapp is connected to the computer, it will be easier to click on the link that will be provided below to send the message. Send the message that will be displayed on the chat box on your Whatsapp application.

If it works you will see a message shown below:

This number that will be displayed here is the “from” number that we will use in our code and the “to” number is your Whatsapp number.

How to send a photo message, using the Twilio service

Copy the following code into your python file.

import os

from twilio.rest import Client

account_sid = os.environ['TWILIO_ACCOUNT_SID']

auth_token = os.environ['TWILIO_AUTH_TOKEN']

client = Client(account_sid, auth_token)

from_whatsapp_number = 'whatsapp:+14155238886'

to_whatsapp_number = 'whatsapp:+254706911425'

message = client.messages.create(body='The engineering project sent your this image!',

media_url='https://www.theengineeringprojects.com/wp-content/uploads/2022/04/TEP-Logo.png',

from_=from_whatsapp_number,

to=to_whatsapp_number)


print(message.sid)

With this now all we have to do is run our app.py program on the terminal.

python app.py

Putting it all together:

So far we have written our motion sensor code, pi camera code, and sound system code, but how can we integrate all these different scripts into one project? Well, let us see how we can do it. As a quick recap, we wanted to detect motion, get the intruder's image and save then send the image to the admin's Whatsapp to alert the presence of an intruder in this project. To do this we have to create another file named main.py and write all the code for those functions in the main file. Copy the code below and paste it into the main file:

import pygame

import RPi.GPIO as GPIO

import time

import picamera

camera = picamera.PiCamera()


GPIO.setwarnings(False)

GPIO.setmode(GPIO.BOARD)

GPIO.setup(11, GPIO.IN) #Read output from PIR motion sensor

GPIO.setup(3, GPIO.OUT) #LED output pin

pygame.mixer.init()

pygame.mixer.music.load("alarm.mp3")

import os

from twilio.rest import Client

account_sid = os.environ['TWILIO_ACCOUNT_SID']

auth_token = os.environ['TWILIO_AUTH_TOKEN']

client = Client(account_sid, auth_token)

from_whatsapp_number = 'whatsapp:+14155238886'

to_whatsapp_number = 'whatsapp:+254706911425'

while True:

i=GPIO.input(11)

if i==0: #When output from motion sensor is LOW

print("No intruders",i)

GPIO.output(3, 0) #Turn OFF LED

pygame.mixer.music.stop()

time.sleep(0.2)

elif i==1: #When output from motion sensor is HIGH

print("Intruder detected",i)

GPIO.output(3, 1) #Turn ON LED

pygame.mixer.music.play()

capture image

camera.capture('intruder.jpeg')

#send image to whatsapp

message = client.messages.create(body='The engineering projects program has detected and intruder!',

media_url='https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Ftse4.mm.bing.net%2Fth%3Fid%3DOIP.q1z1XWRn_WAV4oM-Qr2M2gHaGb%26pid%3DApi&f=1',

from_=from_whatsapp_number,

to=to_whatsapp_number)


print(message.sid)

time.sleep(0.2)

GPIO.cleanup()

break


Captured image

Conclusion

In this article, you learned to build a security system using a motion detector and raspberry pi. We also learned how to set up Twilio to send and receive Whatsapp messages using the Twilio API. This project can be implemented in so many areas therefore it is a good idea for you it plays around with the code and implements some extra features. In the next tutorial, we are going to build a led cube in raspberry pi 4.

Syed Zain Nasir

I am Syed Zain Nasir, the founder of <a href=https://www.TheEngineeringProjects.com/>The Engineering Projects</a> (TEP). I am a programmer since 2009 before that I just search things, make small projects and now I am sharing my knowledge through this platform.I also work as a freelancer and did many projects related to programming and electrical circuitry. <a href=https://plus.google.com/+SyedZainNasir/>My Google Profile+</a>

Share
Published by
Syed Zain Nasir