I am happy to say I have added another 59 entries to the dataset, which added to the 25 that I collected in the previous post brings us to 84 new entries! I was hoping for 100, but that is close enough for me. There are now 313 complete entries.
Dataset:
g3100_E3200_04_16_24.xlsx (Size: 331.59 KB / Downloads: 3)
Reference Images: Ref_images.zip
Let’s see what we’ve learned...
First, we test the new entries against the Fios-F1nDr database. We test 58 entries, since 1 was actually missing the serial. Again it doesn’t look good at first, but all it really means is that we’ve collected a bunch of new Date Codes! After working all of the new entries into the proper place and recalculating everything, we retest against the updated database.
Before:
Correct - 12 (~20.5%)
Incorrect - 23 (~40.5%)
unknown block - 12 (~20.5%)
Unknown device - 9 (~15.5%)
Not Enough Data - 2 (~3%)
After:
Correct - 44 (~75.9%)
Incorrect - 3 (~5%)
unknown block - 0 (0%)
Unknown Device 9 (~15.5%)
Not enough data 2 (~3%)
Much Better! The 3 incorrect are outliers in the dataset, that don’t calculate correctly at the moment. The Unknown devices are showing up because they are either in the DC.F5.1B block or new block 74.90.BC. We are starting to gather a bit more info about both of these, but we need more entries to better understand what’s going on there.
We now have 166 Unique Date codes. There is now a second sheet named “Date Codes” in the dataset that has each of the blocks separated. This eventually gets collapsed into that data_ref_lines that the Fios-F1nDr code uses, which is also included with the dataset. Sorting the entries by the date codes allows us to see devices that are in the same block possibly removing some of the overall randomness. There are quite a few blocks that we have 4-7 entries for.
Also added to the dataset is a new column named “QR”, which indicates if the QR data was read from the reference image. The QR data either reads or it doesn’t, so it is the most accurate and therefore preferred source of the data. This is the script that I am using to sort the scraped images into “found” and “not found”. Found images contain either a QR code, or some part of the back label we are looking for (WIFI, Serial, Password, etc). The script is adapted from code found here, and uses apple vision for the text and zbarimg for the QR code. The script might only work on an Apple devices, but I am not sure. The script will output “imagedata.csv” at the end, that contains all of the info that it finds. The script works great to sort the images, but unfortunately the extracted text is rarely complete or correct, but it’s fun to see what the computer picks up.
Unfortunately, the script is not very good at reading the QR code. Running the script against all of the reference images only yields 38/313 QR codes. Using my phone, with a QR reader app is how I actually collect the QR codes. Using this method I’ve read 133/313 codes, but it takes much longer. If I could somehow get the script to read the QR with the same effectiveness as my phone it would make the scraping process so much faster and simpler! The good news is now that we have the QR column, we actually can separate the reference images into a “Read” and “Unread”, and perhaps use this to train the image processing to better recognize our QR codes. Furthermore, we are looking for a specific format to the QR code, so hopefully this can better inform the script too? Here is what an example read looks like.
If you want to see if your phone is any better at reading images, I can read the first image but not the second.
![[Image: attachment.php?aid=1254]](https://hashcat.net/forum/attachment.php?aid=1254)
![[Image: attachment.php?aid=1256]](https://hashcat.net/forum/attachment.php?aid=1256)
Despite everything that I’ve been able to accomplish, I am very much new to all this. This is my first foray into hardware hacking, and I am working mostly off of tutorials found online. I was able to get binwalk to run, and I can get it loaded into Ghidra, but I honestly have no clue what the next steps should be. I believe it is encrypted, possibly with LUKS. I found an awesome post on the openwrt forums discussing the CR1000A which is a similar device. The post is an exciting read, but quickly got over my head. I also found a nice teardown of the E3200, which seems to be very similar to the G3100 internally. Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP.
"Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP."
I hope that my posts show that I am putting a lot of effort into this, and willing to share my ideas openly. I don’t really expect to find the keygen algorithm, but it’s enough of a carrot to keep me going. I will continue to drudge along by myself, but if you, or anyone you know may be interested in furthering this research please let me know. I have all of the listed firmware downloaded and I’m happy to pass it along.
Dataset:
g3100_E3200_04_16_24.xlsx (Size: 331.59 KB / Downloads: 3)
Reference Images: Ref_images.zip
Let’s see what we’ve learned...
First, we test the new entries against the Fios-F1nDr database. We test 58 entries, since 1 was actually missing the serial. Again it doesn’t look good at first, but all it really means is that we’ve collected a bunch of new Date Codes! After working all of the new entries into the proper place and recalculating everything, we retest against the updated database.
Before:
Correct - 12 (~20.5%)
Incorrect - 23 (~40.5%)
unknown block - 12 (~20.5%)
Unknown device - 9 (~15.5%)
Not Enough Data - 2 (~3%)
After:
Correct - 44 (~75.9%)
Incorrect - 3 (~5%)
unknown block - 0 (0%)
Unknown Device 9 (~15.5%)
Not enough data 2 (~3%)
Much Better! The 3 incorrect are outliers in the dataset, that don’t calculate correctly at the moment. The Unknown devices are showing up because they are either in the DC.F5.1B block or new block 74.90.BC. We are starting to gather a bit more info about both of these, but we need more entries to better understand what’s going on there.
We now have 166 Unique Date codes. There is now a second sheet named “Date Codes” in the dataset that has each of the blocks separated. This eventually gets collapsed into that data_ref_lines that the Fios-F1nDr code uses, which is also included with the dataset. Sorting the entries by the date codes allows us to see devices that are in the same block possibly removing some of the overall randomness. There are quite a few blocks that we have 4-7 entries for.
Also added to the dataset is a new column named “QR”, which indicates if the QR data was read from the reference image. The QR data either reads or it doesn’t, so it is the most accurate and therefore preferred source of the data. This is the script that I am using to sort the scraped images into “found” and “not found”. Found images contain either a QR code, or some part of the back label we are looking for (WIFI, Serial, Password, etc). The script is adapted from code found here, and uses apple vision for the text and zbarimg for the QR code. The script might only work on an Apple devices, but I am not sure. The script will output “imagedata.csv” at the end, that contains all of the info that it finds. The script works great to sort the images, but unfortunately the extracted text is rarely complete or correct, but it’s fun to see what the computer picks up.
Code:
"""
Use Apple's Vision Framework via PyObjC to detect text in images
modified code from: https://gist.github.com/RhetTbull/1c34fc07c95733642cffcd1ac587fc4c
Necessary Dependencies:
python3 -m pip install pyobjc-core pyobjc-framework-Quartz pyobjc-framework-Vision wurlitzer
brew install zbar
This script process all of the images in folder_path using apple vision to extract text.
It checks to see if any of the keywords are found in the image, and extracts the relevent info
All of the info is saved to <file name> and relevent photos are moved to the "found" folder
in case they need to be referenced later. Any photos without keywords are deleted.
"""
import pathlib
import sys
import Quartz
import Vision
import os
import shutil
import pandas as pd
from Cocoa import NSURL
from Foundation import NSDictionary
# needed to capture system-level stderr
from wurlitzer import pipes
import subprocess
import re
from pyzbar.pyzbar import decode, ZBarSymbol
from PIL import Image
sys.path.append(r'/opt/homebrew/Cellar/zbar') #tell the script where to find zbarimg
KEYWORDS = ["Wi-Fi Name:", "Wi-Fi Password:", "Admin URL:", "Admin Password:",
"Model Number:", "Serial #.", "WAN MAC:", "HW ver.:", "Shipped FW ver.:"]
# Define the keyword mappings
qr_to_keyword_map = {
"WIFI:S:": "Wi-Fi Name:",
"T:WPA;P:": "Wi-Fi Password:",
"ROUTER:M:": "Model Number:",
"S:": "Serial #.",
"W:": "WAN MAC:",
"I:admin;P:": "Admin Password:",
}
QRKEYWORDS = ["WIFI:S:", "T:WPA;P:", "I:admin;P:", "ROUTER:M:", "S:"]
KEYWORDS = ["Wi-Fi Name:", "Wi-Fi Password:", "Admin URL:", "Admin Password:",
"Model Number:", "Serial #.", "WAN MAC:", "HW ver.:", "Shipped FW ver.:"]
keywords_lower = [keyword.lower() for keyword in KEYWORDS] #makes keywords lowercase(case insensitive) for better matching
global qr_found
def image_to_text(img_path, lang="eng"):
input_url = NSURL.fileURLWithPath_(img_path)
with pipes() as (out, err):
# capture stdout and stderr from system calls
# otherwise, Quartz.CIImage.imageWithContentsOfURL_
# prints to stderr something like:
# 2020-09-20 20:55:25.538 python[73042:5650492] Creating client/daemon connection: B8FE995E-3F27-47F4-9FA8-559C615FD774
# 2020-09-20 20:55:25.652 python[73042:5650492] Got the query meta data reply for: com.apple.MobileAsset.RawCamera.Camera, response: 0
input_image = Quartz.CIImage.imageWithContentsOfURL_(input_url)
vision_options = NSDictionary.dictionaryWithDictionary_({})
vision_handler = Vision.VNImageRequestHandler.alloc().initWithCIImage_options_(
input_image, vision_options
)
results = []
handler = make_request_handler(results)
vision_request = Vision.VNRecognizeTextRequest.alloc().initWithCompletionHandler_(handler)
error = vision_handler.performRequests_error_([vision_request], None)
return results
def scan_qr(image_path, img_name):
qr_read = pd.DataFrame()
global qr_found
qr_found = False
try:
# Run zbarimg command on the image
#qimage_path = '/Users/Brian/Downloads/Routers/Fios-3100/ebay_images/found/image_702.webp'
img = Image.open(image_path) # Replace with your file
decoded_objects = decode(img)
if decoded_objects:
qr_read = pd.DataFrame(decoded_objects)
#print(decoded_objects)
#print(f"✅ QR Code Found:" + str(decoded_objects))
#qr_codes = [line.split(": ", 1)[1] for line in result.stdout.strip().split("\n") if ": " in line]
#qr_read = pd.DataFrame(qr_codes) # Create a DataFrame
qr_read["QR_read"] = "Yes"
#df = pd.DataFrame(qr_codes, columns=["QR Code Data"])
qr_found = True
#else:
#print(f"❌ No QR code detected in", img_name)
except FileNotFoundError:
print("Error: zbarimg is not installed or not found in PATH.")
#return qr_read
return decoded_objects
def make_request_handler(results):
""" results: list to store results """
if not isinstance(results, list):
raise ValueError("results must be a list")
def handler(request, error):
if error:
print(f"Error! {error}")
else:
observations = request.results()
for text_observation in observations:
recognized_text = text_observation.topCandidates_(1)[0]
results.append([recognized_text.string(), recognized_text.confidence()])
return handler
def extract_values(qrcodes, results, QRKEYWORDS, KEYWORDS):
global match_found
match_found = False
extracted_data = {key: "" for key in KEYWORDS} # Initialize a dictionary with empty values
# Convert QRKEYWORDS to KEYWORDS mapping
qr_to_keyword_map = {
"WIFI:S:": "Wi-Fi Name:",
"T:WPA;P:": "Wi-Fi Password:",
"I:admin;P:": "Admin Password:",
"ROUTER:M:": "Model Number:",
";S:": "Serial #."
}
# **Step 1: Search QR Code Data**
for qr_entry in qrcodes:
data_str = str(qr_entry) # Decode bytes to string
for qr_key, keyword in qr_to_keyword_map.items():
match = re.search(fr'{qr_key}([^;]+)', data_str)
if match:
extracted_data[keyword] = match.group(1).split("'")[0] # Remove trailing artifacts
match_found = True
extracted_data["QR_found"] = "Yes"
# **Step 2: Search OCR Text (Only if not found in QR)**
for text_entry in results:
text = str(text_entry)
for keyword in KEYWORDS:
if extracted_data[keyword]: # Skip if already filled by QR
continue
match = re.search(fr'{keyword}([^;]+)', text, re.IGNORECASE)
if match:
extracted_data[keyword] = match.group(1).split("'")[0] # Extract OCR value
match_found = True
# Convert to DataFrame
extract_df = pd.DataFrame([extracted_data])
return extract_df
folder_path = input("Please enter the file path to folder with the images: ")
folder_path = (folder_path + "/")
folder_contents = os.listdir(folder_path)
os.makedirs(folder_path +"found/", exist_ok=True)
os.makedirs(folder_path +"not_found/", exist_ok=True)
global match_found
match_found = False
matched_text = pd.DataFrame() #the dataframe where well store all of oure matched text
clean_list = pd.DataFrame()
for imgs in folder_contents:
if not imgs.lower().endswith((".png", ".jpg", ".jpeg", ".webp", ".tiff")): #.lower makes the comparison case-insensitive
continue
match_found = False
print ("Processing: " + imgs)
img_path = (folder_path + imgs)
qrcodes = scan_qr(img_path, imgs)
results = image_to_text(img_path)
# Extract values from QR and OCR results
if qrcodes:
#qr_data = extract_values(qrcodes)
print("QR Code Found: " + str(qrcodes))
if results:
#ocr_data = extract_values(results)
print ("OCR Text Found: " + str(results))
if qrcodes or results:
img_data = extract_values(qrcodes, results, QRKEYWORDS, KEYWORDS)
img_data['IMG'] = imgs #add the image name to the info
if match_found == True:
print(f"✅ Match found! Moving: {imgs}")
matched_text = pd.concat([matched_text, img_data], ignore_index=True) #add the extracted info to our dataframe
shutil.move(folder_path + imgs, folder_path + "found/" + imgs) #moves the images to a folder named "found"
else:
print(f"❌ No match. Moving: {imgs}")
shutil.move(folder_path + imgs, folder_path + "not_found/" + imgs) #moves the images to a folder named "found"
#os.remove(img_path) #delete files where no info is found
#Now to do some cleaning to our matched_text
#clean_list['Duplicates'] = 0
if not matched_text.empty:
clean_list = matched_text #copy the matched text into a new dataframe
clean_list = clean_list.fillna("") #remove any nan
clean_list = clean_list.map(lambda x: x.replace(" ", "") if isinstance(x, str) else x)
#clean_list = clean_list.map(lambda x: x.replace("', 0.5]", "") if isinstance(x, str) else x)
img_column = clean_list.pop('IMG') # Removes "IMG" column but keeps it in memory
clean_list["duplicates"] = clean_list.duplicated(keep="first").astype(int) #adds a column "duplicates" and denotes duplicate with 0 or 1
clean_list["duplicates"] = clean_list.groupby(clean_list.columns.drop(["duplicates"]).tolist())["duplicates"].transform("sum") #count the number of duplicates
clean_list = clean_list.drop_duplicates() #.reset_index(drop=True) # Drop duplicate rows and keep the first occurrence
clean_list.insert(0, "IMG", img_column)
clean_list = clean_list[["IMG"] + [col for col in clean_list.columns if col != "IMG"]] # ensures "IMG" is first column, while preserving the rest of the order
clean_list = clean_list.sort_values(by="IMG").reset_index(drop=True) #Arranges list in alphabetical order based on image name
clean_list = clean_list.reset_index(drop=True)
clean_list.to_csv(folder_path + "/imagedata.csv")
else:
print("No matched images found!")
# the number in the results is the confidence score, It's important to note that a lower confidence score doesn't
#necessarily mean the recognized text is incorrect, but it does suggest that the system is less certain about
#its accuracy. In such cases, manual verification or additional processing might be warranted to ensure the information's correctness.Unfortunately, the script is not very good at reading the QR code. Running the script against all of the reference images only yields 38/313 QR codes. Using my phone, with a QR reader app is how I actually collect the QR codes. Using this method I’ve read 133/313 codes, but it takes much longer. If I could somehow get the script to read the QR with the same effectiveness as my phone it would make the scraping process so much faster and simpler! The good news is now that we have the QR column, we actually can separate the reference images into a “Read” and “Unread”, and perhaps use this to train the image processing to better recognize our QR codes. Furthermore, we are looking for a specific format to the QR code, so hopefully this can better inform the script too? Here is what an example read looks like.
Code:
WIFI:S:Verizon_ZFBX9P;T:WPA;P:bread-veto6-ode;;ROUTER:M:G3100;S:G402121090810583;W:3CBDC577D884;I:admin;P:KT7XFBH3D;;4If you want to see if your phone is any better at reading images, I can read the first image but not the second.
Despite everything that I’ve been able to accomplish, I am very much new to all this. This is my first foray into hardware hacking, and I am working mostly off of tutorials found online. I was able to get binwalk to run, and I can get it loaded into Ghidra, but I honestly have no clue what the next steps should be. I believe it is encrypted, possibly with LUKS. I found an awesome post on the openwrt forums discussing the CR1000A which is a similar device. The post is an exciting read, but quickly got over my head. I also found a nice teardown of the E3200, which seems to be very similar to the G3100 internally. Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP.
"Looks like the embedded processor of choice is a Broadcom BCM68369KFEBG, which is a Dual Core 1.5 GHz ARM v8 processor featuring 4x GbE PHYs, 1x USB 3.0, DDR 800MHz support, 4ch VoIP."
I hope that my posts show that I am putting a lot of effort into this, and willing to share my ideas openly. I don’t really expect to find the keygen algorithm, but it’s enough of a carrot to keep me going. I will continue to drudge along by myself, but if you, or anyone you know may be interested in furthering this research please let me know. I have all of the listed firmware downloaded and I’m happy to pass it along.