Verizon Fios G3100 and E3200 Research

[FiosFiend]

I am happy to say I have added another 59 entries to the dataset, which added to the 25 that I collected in the previous post brings us to 84 new entries!  I was hoping for 100, but that is close enough for me.  There are now 313 complete entries.  

Dataset:  
  g3100_E3200_04_16_24.xlsx (Size: 331.59 KB / Downloads: 0)
Reference Images: Ref_images.zip

Let’s see what we’ve learned...

First, we test the new entries against the Fios-F1nDr database.  We test 58 entries, since 1 was actually missing the serial.  Again it doesn’t look good at first, but all it really means is that we’ve collected a bunch of new Date Codes!  After working all of the new entries into the proper place and recalculating everything, we retest against the updated database.

Before:
Correct - 12  (~20.5%)
Incorrect - 23  (~40.5%)
unknown block - 12  (~20.5%)
Unknown device - 9  (~15.5%)
Not Enough Data - 2 (~3%)

After:
Correct - 44  (~75.9%)
Incorrect - 3 (~5%)
unknown block - 0 (0%)
Unknown Device 9  (~15.5%)
Not enough data 2  (~3%)

Much Better!  The 3 incorrect are outliers in the dataset, that don’t calculate correctly at the moment.  The Unknown devices are showing up because they are either in the DC.F5.1B block or new block 74.90.BC.  We are starting to gather a bit more info about both of these, but we need more entries to better understand what’s going on there.

We now have 166 Unique Date codes.  There is now a second sheet named “Date Codes” in the dataset that has each of the blocks separated.  This eventually gets collapsed into that data_ref_lines that the Fios-F1nDr code uses, which is also included with the dataset.  Sorting the entries by the date codes allows us to see devices that are in the same block possibly removing some of the overall randomness.  There are quite a few blocks that we have 4-7 entries for.

Also added to the dataset is a new column named “QR”, which indicates if the QR data was read from the reference image.  The QR data either reads or it doesn’t, so it is the most accurate and therefore preferred source of the data.  This is the script that I am using to sort the scraped images into “found” and “not found”.  Found images contain either a QR code, or some part of the back label we are looking for (WIFI, Serial, Password, etc).  The script is adapted from code found here, and uses apple vision for the text and zbarimg for the QR code.  The script might only work on an Apple devices, but I am not sure.  The script will output “imagedata.csv” at the end, that contains all of the info that it finds.  The script works great to sort the images, but unfortunately the extracted text is rarely complete or correct, but it’s fun to see what the computer picks up.

Code:

"""

Use Apple's Vision Framework via PyObjC to detect text in images
modified code from: https://gist.github.com/RhetTbull/1c34fc07c95733642cffcd1ac587fc4c

Necessary Dependencies:
python3 -m pip install pyobjc-core pyobjc-framework-Quartz pyobjc-framework-Vision wurlitzer
brew install zbar

This script process all of the images in folder_path using apple vision to extract text.
It checks to see if any of the keywords are found in the image, and extracts the relevent info
All of the info is saved to <file name> and relevent photos are moved to the "found" folder
in case they need to be referenced later.  Any photos without keywords are deleted.
"""

import pathlib
import sys
import Quartz
import Vision
import os
import shutil
import pandas as pd
from Cocoa import NSURL
from Foundation import NSDictionary
# needed to capture system-level stderr
from wurlitzer import pipes
import subprocess

import re

from pyzbar.pyzbar import decode, ZBarSymbol
from PIL import Image

sys.path.append(r'/opt/homebrew/Cellar/zbar')  #tell the script where to find zbarimg

KEYWORDS = ["Wi-Fi Name:", "Wi-Fi Password:", "Admin URL:", "Admin Password:",
            "Model Number:", "Serial #.", "WAN MAC:", "HW ver.:", "Shipped FW ver.:"]

# Define the keyword mappings
qr_to_keyword_map = {
        "WIFI:S:": "Wi-Fi Name:",
        "T:WPA;P:": "Wi-Fi Password:",
        "ROUTER:M:": "Model Number:",
        "S:": "Serial #.",
        "W:": "WAN MAC:",
        "I:admin;P:": "Admin Password:",
}

QRKEYWORDS = ["WIFI:S:", "T:WPA;P:", "I:admin;P:", "ROUTER:M:", "S:"]
KEYWORDS = ["Wi-Fi Name:", "Wi-Fi Password:", "Admin URL:", "Admin Password:",
            "Model Number:", "Serial #.", "WAN MAC:", "HW ver.:", "Shipped FW ver.:"]

keywords_lower = [keyword.lower() for keyword in KEYWORDS]  #makes keywords lowercase(case insensitive) for better matching

global qr_found

def image_to_text(img_path, lang="eng"):
    input_url = NSURL.fileURLWithPath_(img_path)

    with pipes() as (out, err):
    # capture stdout and stderr from system calls
    # otherwise, Quartz.CIImage.imageWithContentsOfURL_
    # prints to stderr something like:
    # 2020-09-20 20:55:25.538 python[73042:5650492] Creating client/daemon connection: B8FE995E-3F27-47F4-9FA8-559C615FD774
    # 2020-09-20 20:55:25.652 python[73042:5650492] Got the query meta data reply for: com.apple.MobileAsset.RawCamera.Camera, response: 0
        input_image = Quartz.CIImage.imageWithContentsOfURL_(input_url)

    vision_options = NSDictionary.dictionaryWithDictionary_({})
    vision_handler = Vision.VNImageRequestHandler.alloc().initWithCIImage_options_(
        input_image, vision_options
    )
    results = []
    handler = make_request_handler(results)
    vision_request = Vision.VNRecognizeTextRequest.alloc().initWithCompletionHandler_(handler)
    error = vision_handler.performRequests_error_([vision_request], None)

    return results

def scan_qr(image_path, img_name):
    qr_read = pd.DataFrame()
    global qr_found
    qr_found = False
   
    try:
        # Run zbarimg command on the image
        #qimage_path = '/Users/Brian/Downloads/Routers/Fios-3100/ebay_images/found/image_702.webp'
        img = Image.open(image_path)  # Replace with your file
        decoded_objects = decode(img)
       
        if decoded_objects:
       
            qr_read = pd.DataFrame(decoded_objects)
          #print(decoded_objects)
            #print(f"✅ QR Code Found:" + str(decoded_objects))
            #qr_codes = [line.split(": ", 1)[1] for line in result.stdout.strip().split("\n") if ": " in line]

            #qr_read = pd.DataFrame(qr_codes)    # Create a DataFrame
            qr_read["QR_read"] = "Yes"
            #df = pd.DataFrame(qr_codes, columns=["QR Code Data"])
            qr_found = True
        #else:
            #print(f"❌ No QR code detected in", img_name)
   
    except FileNotFoundError:
        print("Error: zbarimg is not installed or not found in PATH.")
   
    #return qr_read
    return decoded_objects


def make_request_handler(results):
    """ results: list to store results """
    if not isinstance(results, list):
        raise ValueError("results must be a list")

    def handler(request, error):
        if error:
            print(f"Error! {error}")
        else:
            observations = request.results()
            for text_observation in observations:
                recognized_text = text_observation.topCandidates_(1)[0]
                results.append([recognized_text.string(), recognized_text.confidence()])
    return handler


def extract_values(qrcodes, results, QRKEYWORDS, KEYWORDS):
    global match_found
    match_found = False
   
    extracted_data = {key: "" for key in KEYWORDS}  # Initialize a dictionary with empty values
   
    # Convert QRKEYWORDS to KEYWORDS mapping
    qr_to_keyword_map = {
        "WIFI:S:": "Wi-Fi Name:",
        "T:WPA;P:": "Wi-Fi Password:",
        "I:admin;P:": "Admin Password:",
        "ROUTER:M:": "Model Number:",
        ";S:": "Serial #."
    }
   
    # **Step 1: Search QR Code Data**
    for qr_entry in qrcodes:
        data_str = str(qr_entry)  # Decode bytes to string
        for qr_key, keyword in qr_to_keyword_map.items():
            match = re.search(fr'{qr_key}([^;]+)', data_str)
            if match:
                extracted_data[keyword] = match.group(1).split("'")[0]  # Remove trailing artifacts
                match_found = True
                extracted_data["QR_found"] = "Yes"


    # **Step 2: Search OCR Text (Only if not found in QR)**
    for text_entry in results:
        text = str(text_entry)
       
        for keyword in KEYWORDS:
            if extracted_data[keyword]:  # Skip if already filled by QR
                continue
            match = re.search(fr'{keyword}([^;]+)', text, re.IGNORECASE)
            if match:
                extracted_data[keyword] = match.group(1).split("'")[0]  # Extract OCR value
                match_found = True

    # Convert to DataFrame
    extract_df = pd.DataFrame([extracted_data])
    return extract_df

folder_path = input("Please enter the file path to folder with the images: ")
folder_path = (folder_path + "/")
folder_contents = os.listdir(folder_path)
os.makedirs(folder_path +"found/", exist_ok=True)
os.makedirs(folder_path +"not_found/", exist_ok=True)
global match_found
match_found = False

matched_text = pd.DataFrame()  #the dataframe where well store all of oure matched text
clean_list = pd.DataFrame()

for imgs in folder_contents:
    if not imgs.lower().endswith((".png", ".jpg", ".jpeg", ".webp", ".tiff")):  #.lower makes the comparison case-insensitive
        continue
    match_found = False

    print ("Processing: " + imgs)
    img_path = (folder_path + imgs)
    qrcodes = scan_qr(img_path, imgs)
    results = image_to_text(img_path)
   
    # Extract values from QR and OCR results
    if qrcodes:
        #qr_data = extract_values(qrcodes)
        print("QR Code Found: " + str(qrcodes))
    if results:
        #ocr_data = extract_values(results)
        print ("OCR Text Found: " + str(results))
   
    if qrcodes or results:
        img_data = extract_values(qrcodes, results, QRKEYWORDS, KEYWORDS)
        img_data['IMG'] = imgs #add the image name to the info

     
   
    if match_found == True:
        print(f"✅ Match found! Moving: {imgs}")
        matched_text = pd.concat([matched_text, img_data], ignore_index=True)  #add the extracted info to our dataframe
        shutil.move(folder_path + imgs, folder_path + "found/" + imgs)    #moves the images to a folder named "found"
    else:
        print(f"❌ No match. Moving: {imgs}")
        shutil.move(folder_path + imgs, folder_path + "not_found/" + imgs)    #moves the images to a folder named "found"

        #os.remove(img_path)  #delete files where no info is found


#Now to do some cleaning to our matched_text
#clean_list['Duplicates'] = 0
if not matched_text.empty:
    clean_list = matched_text #copy the matched text into a new dataframe
    clean_list = clean_list.fillna("") #remove any nan
    clean_list = clean_list.map(lambda x: x.replace(" ", "") if isinstance(x, str) else x)
    #clean_list = clean_list.map(lambda x: x.replace("', 0.5]", "") if isinstance(x, str) else x)


    img_column = clean_list.pop('IMG')  # Removes "IMG" column but keeps it in memory

    clean_list["duplicates"] = clean_list.duplicated(keep="first").astype(int)  #adds a column "duplicates" and denotes duplicate with 0 or 1
    clean_list["duplicates"] = clean_list.groupby(clean_list.columns.drop(["duplicates"]).tolist())["duplicates"].transform("sum") #count the number of duplicates
    clean_list = clean_list.drop_duplicates() #.reset_index(drop=True)  # Drop duplicate rows and keep the first occurrence

    clean_list.insert(0, "IMG", img_column)

    clean_list = clean_list[["IMG"] + [col for col in clean_list.columns if col != "IMG"]] # ensures "IMG" is first column, while preserving the rest of the order
    clean_list = clean_list.sort_values(by="IMG").reset_index(drop=True) #Arranges list in alphabetical order based on image name
   
    clean_list = clean_list.reset_index(drop=True)

    clean_list.to_csv(folder_path + "/imagedata.csv")

else:
    print("No matched images found!")

# the number in the results is the confidence score, It's important to note that a lower confidence score doesn't
#necessarily mean the recognized text is incorrect, but it does suggest that the system is less certain about
#its accuracy. In such cases, manual verification or additional processing might be warranted to ensure the information's correctness.

Unfortunately, the script is not very good at reading the QR code.  Running the script against all of the reference images only yields 38/313 QR codes.  Using my phone, with a QR reader app is how I actually collect the QR codes.  Using this method I’ve read 133/313 codes, but it takes much longer.  If I could somehow get the script to read the QR with the same effectiveness as my phone it would make the scraping process so much faster and simpler!  The good news is now that we have the QR column, we actually can separate the reference images into a “Read” and “Unread”, and perhaps use this to train the image processing to better recognize our QR codes.  Furthermore, we are looking for a specific format to the QR code, so hopefully this can better inform the script too?  Here is what an example read looks like.

Code:

WIFI:S:Verizon_ZFBX9P;T:WPA;P:bread-veto6-ode;;ROUTER:M:G3100;S:G402121090810583;W:3CBDC577D884;I:admin;P:KT7XFBH3D;;4

If you want to see if your phone is any better at reading images, I can read the first image but not the second.



Despite everything that I’ve been able to accomplish, I am very much new to all this.  This is my first foray into hardware hacking, and I am working mostly off of tutorials found online.  I was able to get binwalk to run, and I can get it loaded into Ghidra, but I honestly have no clue what the next steps should be.  I believe it is encrypted, possibly with LUKS.  I found an awesome post on the openwrt forums discussing the CR1000A which is a similar device.  The post is an exciting read, but quickly got over my head.  I also found a nice teardown of the E3200, which seems to be very similar to the G3100 internally.  Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP.

"Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP."

I hope that my posts show that I am putting a lot of effort into this, and willing to share my ideas openly.  I don’t really expect to find the keygen algorithm, but it’s enough of a carrot to keep me going.  I will continue to drudge along by myself, but if you, or anyone you know may be interested in furthering this research please let me know.  I have all of the listed firmware downloaded and I’m happy to pass it along.