AimBuddy

On-device computer vision and control for aiming in touchscreen 3D shooters.
It finds people in the game picture, marks them, and steers the view toward the nearest one with one extra finger.

How it works · Steering the camera · Target marks · Detection · Touch helper · Benchmarks · Install

AimBuddy is built for players with low vision or motor impairments, for whom fast, precise aiming on a touchscreen is the barrier. It reads the game's picture, finds people in it with a neural network (or, as an experiment, by the colour of the outline a game draws round them), draws marks over them, and, once the player switches steering on, drags one extra finger across the screen to turn the camera toward the nearest one. The player's own fingers keep working at the same time.

Everything runs on the phone. The app's only network use is the model store and the update check, both on GitHub. Free and open source.

What is technically different

How it works

AimBuddy is three programs: the app (Kotlin), a native library in the same process (C++), and a touch helper in a process of its own (libaimtouch.so, C++) that Shizuku or root starts. The map below shows where each piece lives and what flows between them.

flowchart LR
    subgraph app["App process (Kotlin)"]
        projection["MediaProjection<br/>ImageReader"]
        windows["Overlay windows"]
    end

    subgraph native["Native library (C++17)"]
        roi["RoiCapture<br/>square round the crosshair"]
        mailbox[("LatestMailbox<br/>newest frame wins")]
        detector["Detector<br/>model or colour"]
        look["GlobalMotion<br/>camera look"]
        camera[("CameraMotion<br/>finger log, measured slides,<br/>learned gain and delay")]
        tracker["Tracker<br/>Kalman filters"]
        aim["AimController<br/>a step every 8 ms"]
        render["ImGui renderer"]
    end

    subgraph helper["Touch helper (own process)"]
        mux["TouchMux"]
        virtual["Virtual touchscreen"]
    end

    real(["Real touchscreen"]) --> mux
    projection --> roi --> mailbox --> detector --> tracker
    roi --> look --> camera
    camera -->|"camera slide removed<br/>from target speed"| tracker
    tracker --> aim
    camera <-->|"every move logged,<br/>gain and delay read"| aim
    aim -->|"extra finger,<br/>loopback socket"| mux
    mux --> virtual --> game(["Game"])
    tracker --> render --> windows

One frame, from the screen to the finger and to the overlay:

sequenceDiagram
    autonumber
    participant S as Screen
    participant C as Capture thread
    participant P as Pipeline thread
    participant T as Tracker
    participant A as Aim loop
    participant H as Touch helper
    participant O as Overlay thread

    S->>C: a frame (only when the screen changed)
    Note over C: copy the square round the crosshair
    C->>P: publish to the mailbox (an unread frame is replaced)
    Note over C: camera look on the same frame, about 30 a second
    Note over P: detect with the model or by colour
    P->>T: update(boxes, captureNs, arrivalNs)
    loop every 8 ms
        A->>T: snapshot(now) and snapshot(now + learned delay)
        Note over A: pick a target, plan the finger step
        A->>H: AimDown, AimMove or AimUp (16-byte packet)
    end
    Note over H: merge with the player's fingers into the virtual touchscreen
    loop every screen refresh
        O->>T: snapshot(now + 20 ms)
        Note over O: draw the marks, or skip a frame that looks like the last
    end

Three rules keep the delay short. The newest frame wins: a slow phone drops frames and a result is never older than one detection. The camera look runs after the frame has been published, so the detector never waits for it. And every step plans through the delay it knows about: the aim controller aims where the target will be when its move takes effect, and the overlay draws 20 ms ahead.

classDiagram
    direction TB
    class DetectionPipeline
    class RoiCapture
    class LatestMailbox
    class Detector {
        <<interface>>
        detect(frame)
    }
    class NcnnDetector
    class TfliteDetector
    class ColourBlobFinder
    class GlobalMotionEstimator
    class Tracker
    class CameraMotion
    class ResponseEstimator
    class EgoMotion {
        <<interface>>
        shiftBetween(fromNs, toNs)
    }
    class AimLoop
    class AimController
    class TouchLink
    class TouchServer
    class TouchMux
    class EventSink {
        <<interface>>
        emit(type, code, value)
    }

    DetectionPipeline *-- RoiCapture
    RoiCapture *-- LatestMailbox
    DetectionPipeline *-- Detector
    NcnnDetector ..|> Detector
    TfliteDetector ..|> Detector
    DetectionPipeline *-- ColourBlobFinder
    DetectionPipeline *-- GlobalMotionEstimator
    DetectionPipeline *-- Tracker
    DetectionPipeline --> CameraMotion : records slides
    CameraMotion *-- ResponseEstimator
    CameraMotion ..|> EgoMotion
    Tracker --> EgoMotion : removes camera motion
    AimLoop *-- AimController
    AimLoop --> Tracker : reads targets
    AimLoop --> TouchLink : sends moves
    AimController --> CameraMotion : logs moves, reads the response
    TouchLink ..> TouchServer : packets over loopback
    TouchServer *-- TouchMux
    TouchMux --> EventSink : writes the merged events

Steering the camera

How far a game's view turns for each pixel of finger movement (the gain) differs between games and sensitivity settings by a factor of ten or more, and the game shows the turn a frame or two late (the delay). A controller that guesses either figure wrong overshoots and circles the target, or crawls. AimBuddy asks the player for neither. The controller records every move it makes, a second stage measures how the whole picture slid, and a fit of the one against the other gives the gain and the delay. Ordinary aiming is the experiment.

flowchart LR
    tracker["Tracker<br/>targets and their speed"] --> controller["AimController"]
    controller -->|"finger move"| game["Game camera<br/>gain and delay unknown"]
    game -->|"the scene slides"| look["GlobalMotion<br/>phase correlation"]
    look --> log[("CameraMotion<br/>moves and slides")]
    controller -->|"records every move"| log
    log --> estimator["ResponseEstimator<br/>least squares"]
    estimator -->|"gain, delay, confidence"| controller
    log -->|"how far the scene slid"| tracker

Every 8 ms the controller does four things:

1. It plans through the delay. A move shows up on screen one or two frames later, so judging by the screen as it is now would order the same correction again and again. The controller asks the tracker where each target will be at now plus the learned delay, with the camera slide that earlier moves have already promised included, and aims there.
2. It leads a moving target. The tracker's speed estimate has the camera's own movement taken out. A target running at speed v needs the scene to slide at v too, so the finger keeps moving at v / gain even when the error is zero. The lead fades in as the speed rises above its own uncertainty, and fades out while the camera cannot be read.
3. It closes the gap and does not chase it. Each step removes a fraction of the remaining error, 1 - exp(-dt / tau), with tau from 200 ms at strength 0 down to 25 ms at strength 1, under a finger speed limit and with a dead zone round the aim point so detector noise does not make the finger twitch.
4. It is careful while the camera is unknown. Until the gain is pinned down, only 8 + 300 * confidence^3 pixels of finger travel may be on their way at once, so the first move is small and its effect teaches the real gain. If the aim swings anyway (three zero crossings within 0.7 s), a stability factor takes the strength down by 40% each time, to a floor of 20%, and gives it back once the aim has been calm for half a second.

How the finger behaves over time, including the cases where the game or the detector misleads it. The waits and the times are in the dropdowns below.

stateDiagram-v2
    direction LR
    [*] --> Idle
    Idle --> Reaching: target in reach
    Reaching --> Waiting: game ignores the finger
    Reaching --> HandsOff: one correction made
    Reaching --> GivenUp: target stuck to the screen
    Waiting --> Idle: wait over
    HandsOff --> Idle: lock moves on
    GivenUp --> Idle: time passes
    Reaching --> Idle: circle's edge, target lost

Sxf(d) = sum of slide * finger(d)       Sff(d) = sum of finger(d)^2       Sss = sum of slide^2

finger(d)  the finger travel that landed in the slide's window if the game takes d to answer
gain(d)    = -Sxf(d) / Sff(d)           the scene slides the opposite way to the finger
miss(d)    = Sss - Sxf(d)^2 / Sff(d)    what a straight line through the points leaves unexplained
delay      = the d with the smallest miss, refined by a parabola through its two neighbours

Target marks

The overlay is a window of marks over the game: where the people are, what the controller is looking at and how far it reaches. Most marks have a switch in the app and in the in-game menu.

The marks are placed with the same tracker the aim uses. Targets are projected 20 ms ahead of now, so a mark lands where the target will be when the frame reaches the glass. A new target is drawn from its third sighting on, or at once when the detector is at least 90% sure, so a box reported for a single frame is never shown. A target that goes unseen for a moment keeps moving along its path and fades, so a mark does not blink because of one missed detection. Boxes the aim controller has given up on as decoys are not drawn.

sequenceDiagram
    participant P as Player
    participant C as OverlayController
    participant S as SecureShield
    participant R as ImGui renderer

    P->>C: taps the floating button
    C->>S: raise a see-through secure window over the menu's column
    Note over C,S: wait four frames
    C->>R: draw the menu
    Note over S: a recording shows black here and the game everywhere else
    P->>C: closes the menu
    R-->>C: menu cleared
    Note over C,S: keep the shield a few frames, then remove it

Detection

The detector reads a square round the crosshair, between 1 and 1.75 times the model's input size (256 to 448 px for the standard model), copied out of the capture buffer inside the callback because the buffer is recycled the moment it returns. The newest frame wins, so what the pipeline reports is never older than one detection.

flowchart LR
    start["Press Start"] --> choice{"Find targets<br/>with"}
    choice -->|Colour| colour["Colour finder<br/>no model to load"]
    choice -->|Model| time["Time the model on the CPU and on the GPU<br/>3 warm-up runs, 12 timed, median"]
    time --> faster{"GPU under 80%<br/>of the CPU's time?"}
    faster -->|no| cpu["Use the CPU"]
    faster -->|yes| agree{"GPU answers a test<br/>picture like the CPU?"}
    agree -->|no| cpu
    agree -->|yes| gpu["Use the GPU"]

The GPU is shared with the game, so it has to be clearly faster. It also has to give the CPU's answer to a test picture (the sorted values of the two outputs within 5% of the CPU's range at 64 ranks, and none that is not a number), because a driver that is fast and wrong would leave the detector blind. While Android reports that the phone is warm, detection is limited to 30 runs a second, and to 15 when it is hot. FrameThinner keeps to that schedule by skipping frames, not by holding them back, so a frame that is detected is detected the moment it arrives.

A model comes in one of two formats, and only the part that runs the network differs. An NCNN model (.param and .bin) runs on the CPU or through Vulkan. A TFLite model (one .tflite file) runs through LiteRT 1.4.2: on the CPU through XNNPACK, or through LiteRT's GPU delegate (OpenCL, half precision), chosen by the same timing and test picture. The model says how it wants its picture, channels per pixel or in planes, floats or 8-bit integers, and the detector writes exactly that straight into the model's input with one table look-up per value, after the same resize the NCNN path uses. Decoding, tracking and everything after are shared. The LiteRT libraries are opened only when a .tflite model is loaded, so an NCNN model never loads them.

stateDiagram-v2
    direction LR
    [*] --> Tentative: unmatched detection of 50% or more
    Tentative --> Tracked: three matches
    Tentative --> [*]: two misses in a row
    Tracked --> Coasting: missed
    Coasting --> Tracked: matched again
    Coasting --> Lost: unseen for 350 ms
    Lost --> Tracked: a confident box on its path within 1 s
    Lost --> [*]: after 1 s

flowchart LR
    frame["Square round<br/>the crosshair"] --> mask["Mask<br/>65 536-entry table"]
    mask --> close["Close<br/>small gaps"]
    close --> label["Label<br/>the pieces"]
    label --> join["Join pieces<br/>that lie close"]
    join --> shape["Keep patches<br/>shaped like a person"]
    shape --> tracker["Tracker<br/>one class, score 0.9"]

Touch helper

Android's input dispatcher follows one input device at a time in a window. When a second device starts a touch while the first has a finger down, the first one's moves are dropped. So the extra finger cannot come from a second virtual touchscreen next to the real one. The touch helper instead takes the real touchscreen over, passes every event it reports to one virtual copy of it, and adds the extra finger to that copy as an extra contact in the next free slot. The game sees one screen with the player's fingers plus one more.

sequenceDiagram
    participant App as App (TouchLink)
    participant Helper as Touch helper
    participant Real as Real touchscreen
    participant Virtual as Virtual touchscreen

    App->>Helper: start through Shizuku or root, with a secret token
    Helper->>Real: open it (not yet taken)
    Helper->>Virtual: create it, named like the real one plus "-ab", with one slot more
    App->>Helper: Hello ("ABT1", version 1), then the token
    Helper-->>App: Ready (slot count, raw ranges)
    App->>Helper: Configure (display size, rotation)
    App->>Helper: Engage
    Note over Helper: waits until no finger is on the screen
    Helper->>Real: grab it
    Helper-->>App: Merging
    loop heartbeat every 100 ms, aim steps every 8 ms
        App->>Helper: Ping, AimDown, AimMove, AimUp
        Real->>Helper: the player's events
        Helper->>Virtual: one merged stream
        Helper-->>App: Player (fingers down, speeds)
    end
    App->>Helper: Release
    Helper->>Real: give it back

Everything that could leave the player without a working screen is guarded:

• The screen is taken over only when no finger is on it, and the app waits 700 ms after a takeover or a screen rotation before the first touch of the extra finger, because the system ignores a touch that begins sooner.
• The helper needs a heartbeat. If the app is silent for 700 ms the screen is given back at once, and after 5 s of silence the helper exits.
• Holding both volume keys for 1.5 s gives the screen back and stops the helper.
• If the helper is killed, the kernel gives the screen back by itself.

stateDiagram-v2
    direction LR
    [*] --> Released: started, virtual touchscreen created
    Released --> WaitingForFingers: Engage while fingers are down
    Released --> Merging: Engage with no finger down
    WaitingForFingers --> Merging: the last finger lifts
    Merging --> Released: Release, heartbeat lost, volume keys, touchscreen lost

Models and updates

The app carries no model: the APK is about 10.6 MB, 2.9 MB of it LiteRT for .tflite models, and a first run opens on the Models page, where AimBuddy General (9.2 MB) is one tap away. The store is a static model/index.json served from this repository, listing each model's files with their size and SHA-256. Every file is checked against those figures, a model replaces the old one only when it is complete, and a failed or cancelled download leaves the working model alone. A pair of .param and .bin files, or one .tflite file, can be imported too, and is kept only if the detector can load it. docs/Training.md says how to export a .tflite model that keeps up with the game. See model/README.md.

AimBuddy General 2 is made for phones first. It learned from Free Fire and PUBG Mobile footage and from four PC shooters, and it has a class for the player's own character, which the app leaves out. It reads the same 256 px square as version 1 and takes as long (8.1 ms a frame against 8.6 ms on a laptop processor, the two run in turn). It was judged on 602 views from Free Fire and PUBG Mobile videos it never saw, with 57 players in them counted by eye. At the default confidence it found 30 of the players, where version 1 found 13 and sunxds 17. It boxed the player's own character in none of the views, where version 1 did in 111, and it drew 6 boxes on things that are not people (a pylon, a distant car and a figure painted on a car), where version 1 drew 2. On the test pictures of four PC sets, which come from the same games it trained on, it found 86% of the players in CS:GO, 78% in Apex Legends, 97% in Halo Infinite and 68% in Far Cry 6, where version 1 found 18%, 5%, 0% and 58%. docs/Training.md says how it is built and judged.

The app also reads the project's GitHub releases and offers a newer build of itself, and installs it only if it passes every check:

flowchart LR
    feed["Releases list<br/>api.github.com"] --> newer{"A newer version<br/>for this build?"}
    newer -->|no| current["Up to date"]
    newer -->|yes| download["Download the APK"]
    download --> checks{"SHA-256 as listed,<br/>same package,<br/>higher version,<br/>same signing certificate?"}
    checks -->|no| refuse["Refuse, link to the releases page"]
    checks -->|yes| installer["Hand it to the system installer,<br/>the player confirms"]

Benchmarks

Three kinds of figure follow, and each says where it was measured. No phone has been benchmarked for this README: the performance readout (Settings, Overlay) and the Info tab of the in-game menu show the figures for yours, and docs/Performance.md says how to read them.

Per-frame steps on a computer

sh app/src/test/native/bench/run.sh builds the core with the Android build's optimisation flags (-O3 -fno-math-errno -funroll-loops) through zig c++ 0.16.0 and times each step on one thread. These figures are from an Intel Core i5-12500H laptop, x86-64, so they show the work a step does and not how long a phone takes. Each figure is the middle of three runs of 200, median and 95th percentile; a median moved by up to 13% between the three runs and by up to 26% against a set taken on another day, so read them as sizes.

Apart from the detector, which depends on the model and the phone, the camera look is the one step that takes a noticeable share of a frame (16.7 ms at 60 Hz). It runs after the frame has been handed to the detector, and at most about 30 times a second. The tracker update costs about twice what it did with one filter per axis (0.57 µs), because each axis now runs two filters and mixes them. Within the detector, on this processor through NCNN's Python binding with three threads, running AimBuddy General 2 takes about 8 ms, turning the square into the model's input 0.3 to 0.6 ms, and decoding the output 5 µs, so the model is nearly all of it. A TFLite model's input is written in 36 to 63 µs at 256 px, so there too the model is nearly all of it.

Learning an unknown camera, in simulation

The aim controller runs against the simulated game that the tests use: a still target 260 to 280 px from the crosshair, a detector at 30 Hz with 2 px of noise, 10% dropouts and 15 ms of latency, and a camera measurement with 0.7 px of noise. The controller is told 1.5 px of slide per finger pixel and a 50 ms delay, and the simulated game uses the gain and delay in the first two columns. "Guess kept" is the same controller with Calibrate automatically off. Settling is the first moment the error stays under 8 px for good, within a run of 4 s, and overshoot is how far the aim went past the target. Each row is five runs with different detector noise: settling and what was learned are their averages, "2 of 5" says how many runs settled at all, and overshoot is the largest of the five. The runs are seeded, so sh app/src/test/native/bench/run.sh camera prints the same table every time.

With learning, the aim settles in 0.6 to 1.0 s across a 30-fold range of gains and a 10-fold range of delays, without overshoot up to a delay of 110 ms; at 150 ms the worst of the five runs overshoots by 160 px. The learned gain is within 4% of the true one and the learned delay is 2 to 5 ms above the true one. A controller that keeps its guess overshoots by 67 to 1184 px in every row where the real gain is 2.5 or more, settles in only 2 of 5 runs at a gain of 8 and never at 12, crawls (2.0 and 3.4 s) at gains of 0.6 and 0.4, and needs 1.8 to 3.9 s when the delay is 80 ms or more. When the guess happens to be right (gain 1.5, delay 50 ms) learning costs 0.12 s, because the first moves are small on purpose. This is a simulation: it shows the controller and the estimator doing what they are built to do, and not how any game behaves.

Changes that were measured

Each change below was measured when it was made, and the figures are in the changelog. The Android emulator runs the app's arm64 code through translation on a laptop, so its timings are not a phone's.

Where the delay comes from

Install

1. Download AimBuddy-v<version>.apk from Releases and open it. The -debug file beside it is a debuggable build for testing and bug reports; you do not need it. The app installs over 0.3.0-beta.2 and later without uninstalling. If Android says the package conflicts with an existing one, uninstall the old copy once and install again, see Troubleshooting.
2. Open AimBuddy. It opens on the Models page: tap Get on AimBuddy General, or import a model of your own. The app carries no model, so this comes first.
3. When it asks, allow display over other apps, notifications and screen capture.
4. To let AimBuddy steer the camera, give it touch access with Shizuku (no root needed) or with root, and check it with Test touch access on the Touch access screen. Without either, AimBuddy still outlines people.
5. Press Start, then switch to your game. Switch steering on from the second tab of the in-game menu once you are in the game.

Releases before 0.3.5-beta.1 cannot update themselves, so install that one by hand; later ones arrive in the app.

Build and test

You need JDK 17, the Android SDK 35 with build tools 36, NDK 29.0.13113456 and CMake 3.22.1.

./gradlew assembleDebug                 # app/build/outputs/apk/debug/app-debug.apk
./gradlew testDebugUnitTest             # Kotlin tests
sh app/src/test/native/run.sh           # C++ core tests, with a C++17 compiler on the path
sh app/src/test/native/bench/run.sh     # the benchmarks above

• app/src/test/native holds C++ tests of the parts that perform no input or output: the tracker, camera motion, the touch merger, the detector's decoding, the colour finder, the menu's drag scroller, the overlay's fingerprint and the settings. The aim controller is run against a simulated game with a known camera gain and delay, and has to settle for gains and delays it was not told.
• app/src/test/java holds Kotlin tests of the update logic, the model store (downloads, cancelling, failures), the touch access checks and the thermal limits.
• training/tests holds tests of the training pipeline's dataset split and of the labelling rules of the universal model's pipeline (training/universal).
• Debug builds include a stand-in first-person game with a known camera response, used to run the whole loop on an emulator.

Each build generates its own launcher icon from a random seed; -Paimbuddy.iconSeed=<number> repeats one. Releases are described in docs/Releasing.md.

Documentation

Credits

NCNN and LiteRT run the models and Dear ImGui draws the in-game menu. Shizuku provides touch access without root. The app is built with Jetpack Compose and the Geist font. The detection models are made with Ultralytics YOLO. AimBuddy General learned from video labelled by sunxds, a model from sunone_aimbot by SunOner, and from the CS:GO, Apex Legends, Halo Infinite and Far Cry 6 sets of Roboflow 100 (CC BY 4.0); docs/Training.md lists every source. The full list, with licenses, is under Settings, About, Open source notices.

License

AimBuddy is released under the AimBuddy Community Free Use License v1.0. In short: use it, change it and share it freely; do not sell it; anything you build from it stays free under the same terms and says "Based on AimBuddy" with a link here. See LICENSE for the full terms.

AimBuddy General, the detection model the store offers, is trained with Ultralytics YOLO26 and is licensed under AGPL-3.0.

Legal notice

AimBuddy helps with aiming. Use it only where local law, the game's rules and the platform's terms allow it. Some games forbid help with aiming in online play, and using it there can get an account banned. You are responsible for how you use it.

Colour detection that needs no model, two choices for what steering does on a target, a Touch access screen with a test button, and the fixes that five independent reviews of the code turned up.

Added

• Two aim choices, in the app's Settings and in the in-game menu (English, Chinese and Russian). "Once on target" is what steering does when the crosshair is on a target: keep following it, as before, or make one correction, lift the finger and leave the aiming to the player until the next target (or until the same one has left the reach and come back). In the simulation the one correction landed on a still target 300 px away in under a second, and it did not follow a target swinging 140 px to either side afterwards, which is the point of it. "Switching targets" is how readily the lock moves to a target that has come closer to the crosshair: Sticky (the rival must be 45% closer and stay so for 0.3 s), Balanced (30% closer for 0.15 s, as before) or Quick (15% closer for 0.06 s, which moves from one player to the next almost at once).
• Colour detection, a way to find targets with no model (experimental). Under Settings, Detection, choose Colour for Find targets with, and press Start. Open the in-game menu, press Learn colour and tap the colour of an enemy's outline inside the blue square. AimBuddy takes the most vivid colour round the tap and looks in the square for patches of that hue (the tolerance is 14 degrees and can be set from 4 to 40). It closes the small gaps that scenery makes in an outline, joins the head and the body of one player, and keeps the patches that are shaped like a person and at least 40 screen pixels tall (10 to 300 can be set). Every patch is reported at 90% and tracked like a model's box. The overlay turns its own marks away from the colour looked for, because they are in the picture too, and the performance readout says Colour where it says CPU or GPU. A colour that is also in the scenery or on the game's buttons gives false targets. On the development emulator the finder took 6.6 to 9.8 ms a frame over a minute, and its targets were 20 to 28 ms old when they were ready.
• A Touch access screen, opened from the Touch access row on Home and from Settings, Touch (English, Chinese and Russian). It holds the choice of route (Automatic, Shizuku or root), shows where Shizuku stands (Ready, Tap to allow, Not running or Not installed, each with the tap that does the next step) and whether root has been allowed, which is only known once it has been asked for, and has a Test touch access button. The test starts the touch helper by the chosen route the way Start does, says "Works through Shizuku" (or root) or why not, and stops the helper again; it is off while AimBuddy runs, and a press of Start is refused with a message until it has finished. It replaces the dialog that the Home row used to open, and the choice of route moved from Settings to this screen. On the emulator the test passed through Shizuku with the choice Automatic and with the choice Shizuku, said "Root was not granted" for root (the app is not allowed to run su there), and left no helper process behind.
• A download from the store shows how much has arrived, how fast and how long is left, and can be cancelled at any moment: the read ends at once, even on a connection that has gone quiet, and nothing is left behind. A download that fails says why under the model (no connection, a damaged file, no space) and offers Try again, where it used to say so once in a message that went away. When the store cannot be reached, the list saved by the last visit is shown with a note that it is old.
• A benchmark program, sh app/src/test/native/bench/run.sh. It times the per-frame steps (the colour finder, the camera look, the tracker, one step of the aim controller, the touch merger and the overlay's fingerprint) and runs the aim controller against simulated games whose camera gain and delay differ from what it was told. The README shows its figures.

Changed

• Streamer mode no longer blacks out the whole screen while the menu is open. The secure window now covers only the menu's column of the display (see-through, as wide as the menu with a margin, as tall as the screen), and the full screen overlay window is never secure. On an emulator, a screenshot taken with the menu open was 49.5% solid black in landscape (2400x1080, in x 136 to 1320) and 80.4% in portrait (1080x2400, where the menu fills most of the width), and the rest was the game; the earlier design gave 97.3% black in portrait. With the menu closed nothing is black and nothing of AimBuddy is drawn: no boxes, no outlines for the fields of view, no touch area, no labels, no readout, no button. Streamer mode also no longer draws the guides while the menu is open, the menu cannot be resized in it, and "Move or resize on screen" and "Learn colour" are dimmed with a note, because they draw over the game and the colour is read from the screen. If the system refuses the secure window, the menu stays shut and a message says why. An open menu is closed when the screen turns.
• The detector gets a frame before the camera motion is measured on it, and no longer waits for it. The measurement (the camera look, at most 30 a second) ran on the capture thread before the frame was handed over, so every frame it was made on reached the detector late by as long as it took. On the emulator, in a scripted scene where the extra finger turns the camera the same way every time, the frame age (from the frame being on screen to its result being ready) was 37.3 and 32.4 ms in two runs before the change and 12.0 and 16.7 ms in two runs after it, while a look took 1.5 to 21 ms on average in those runs. The emulator runs the app's arm64 code through translation, so these are not a phone's figures.
• The camera look is about 28% faster. It compared the length of a gradient with a library call (hypot) where the squares give the same answer, and it read five neighbouring samples in each step of its refinement from five separate places where one set of pixels serves. Replaying 778 looks recorded on the emulator through the estimator on a desktop processor, fastest of seven passes for each: 0.84 ms on average before, 0.61 ms after, and the slowest 4.0 ms before and 2.2 ms after. The answers differ by at most 0.00001. The log line Camera: says how long a look took, on average and at most.
• The overlay leaves a frame out when it looks like the last one it drew. The screen capture only delivers a frame when the screen changes, and the overlay is on the screen, so an overlay that redrew itself 60 times a second kept the detector running on a still picture. On the emulator, with AimBuddy running over the app's own home screen, the app's threads used 214% and 203% of a core in two runs before the change and 46.6% and 2.6% in two runs after it (the readout was still settling in the first). A moving game is not affected: its frames change.
• The in-game menu was redrawn. Sliders show their name and value above a thin track, and a drop down list has its name above it. The tabs stay in view while a page scrolls, and a page scrolls when you drag a finger up or down on it (a finger that moves sideways still drags a slider). The page buttons are pills as wide as the page, and Close and Stop AimBuddy are two full width pills under a divider, Stop in red. The colour detection settings are in a card, and the prompt for learning a colour is a rounded bar in the menu's colours, with a dot that shows whether the colour was found.
• Settings fit more on a phone held sideways. In a window shorter than 480 dp the rows sit closer (a slider row is 72 dp tall instead of 87, a choice row 89 instead of 99, a switch row 49 instead of 56), the gap between groups is 21 dp instead of 34, and the top bar slides away as the page scrolls forward and comes back at the first swipe the other way. A window about 1100 dp wide or more gets a third column. The ends of a slider's track now sit under the ends of the text above it, where they had been 15 dp inside. Measured on screenshots of the emulator at 2400 by 1080 (the third column at 1200 dp wide); portrait is unchanged.

Removed

• The Touch access list in the in-game menu. It changed a setting that is only read when AimBuddy starts, so it did nothing while AimBuddy ran. It is on the app's Touch access screen.

Fixed

• Pressing Stop within a few seconds of Start crashed the app when Shizuku was running: Stop interrupts the thread that waits for Shizuku to start the touch helper, and nothing caught the interruption. The wait now gives up quietly and the stop finishes. Found in 0.3.5-beta.1.
• Start did nothing for a player who had said no to the notification permission twice: the app asked again as fast as the system answered (50 times in 4 seconds on an emulator) and never reached the screen capture step. It asks once for each press of Start now.
• A model whose description file was unreadable or gave a network input of an absurd size could take the app down while it started. A size outside 32 to 2048 pixels is now left for the detector to work out, an unreadable description is ignored, and a failure while the model or the touch helper starts ends in a message instead of a crash.
• The model store showed an older list, or said it could not be read, when the list had been fetched fine but the copy kept for offline use could not be written (a full phone). The fresh list is used and only the copy is skipped.
• A download and an import that finished at the same moment could leave one of the two models off the list, though its files were there. Changes to the list are made one at a time now; a test that adds 300 models from two threads loses some without it.
• Starting the touch helper through Shizuku again soon after it had been stopped could hang for the whole 10 second wait and fail (the second of two restarts 12 seconds apart did, on an emulator). Shizuku keeps one record for each service, and the new start joined the one that was still being taken down. Each start asks for a service of its own now.
• The Home screen went on showing the touch helper as connected after it had gone away (the system stopped it, the touchscreen's device vanished, or the player held both volume keys). The app now notices within half a second, starts the helper again up to three times a minute (not after the volume keys, which the player pressed on purpose), and then says why steering is not available. Killing the helper on an emulator three times in 40 seconds brought it back each time in about three seconds.
• A damaged line in settings.cfg cost its setting more than its value: an empty or text value was read as 0 (10% detection confidence, camera sensitivity 0.1, a switch turned off) instead of keeping the default, nan got through the range limits to the aim, and a number too big for its type wrapped round to a valid-looking one. Such lines are ignored now, and a file whose last write failed is no longer put in place of the good one.
• A game that took no notice of the extra finger was touched again and again when the finger kept reaching the edge of its touch area. Each new touch started the wait for an answer over, so the game was never judged to be ignoring it: in a 20 second simulation the finger went down 172 times and was never lifted for a wait. The wait is counted over the whole run of touches now, and the same run ends with 6 waits that grow and 90 touches.
• The training pipeline split its frames into train, validation and test by hashing each frame's name, so frames cut from the same video, which look almost the same, landed on both sides. The validation score and the choice of best.pt then measured how well the model remembered its training frames. The split hashes the name of the video now and keeps all of its frames together. A test with 200 videos of 6 frames each finds every video in one split (it does not with the old rule). A dataset that was split by frame before keeps the frames it has where they are, and the frames added later follow their video. With only a few videos the validation split can come out empty, because each video goes to one split: the script says so and stops (training needs a validation set), and --by-frame gives the old frame by frame split for a tiny dataset, with an optimistic score.
• Switching steering off and on again, or turning the screen, kept the wait the game had earned by ignoring the finger before. After four ignored fingers the next touch was two seconds away, and stayed so across the restart. A restart forgets it now.
• The performance readout went on showing the last detection rate after the screen stopped changing, and a pause pulled the average down only slowly (a 10 second pause counted as one reading of 0.1 a second among the 20 or so that make up the average). It reads 0 once no result has come for a second, and the average starts again with the next results. On the emulator the readout said 0/s 25 seconds after AimBuddy was started over a still screen.
• Two equal frames of a picture with no edges in it (the sky, a plain wall, a view filled by one enemy) were taken as proof that the camera had not moved, with full trust. In front of a plain wall the camera could turn while the measured slide stayed zero, and the finger could be judged ignored for it. Such a pair of frames earns trust in proportion to the edges it holds now, and none when the picture is flat.
• A touch area placed near the edge of the screen could be larger than the room it had. The touch helper holds a finger at the edge when it is sent past it, while the controller counted the whole move, so the camera measurements and the check for an ignored finger saw a turn that never happened. The area is cut back to the nearest edge now, and the overlay and the editor draw the circle the controller really uses (76 px for an area at 93% of the width of a 1080 px screen).
• The controller pulled at full strength while the player turned the camera by hand, so the player's turn and the extra finger's pull added up. A setting for giving way already existed, but nothing told the controller when the player was turning the camera. The touch helper now reports how fast the player's fingers move (speeds only, never positions), and while the finger on the same half of the screen as the extra finger's resting place is turning the camera, the controller closes the gap at 40% of its strength and takes it back as the finger stops (it falls away over about a third of a second). On an emulator, a swipe injected through the touchscreen device was noticed 0.4 s after it began; a swipe on the other half of the screen and a finger resting on the look area were not noticed at all.
• A tap on an open list of the in-game menu, such as the language list, closed the menu instead of choosing when it landed past the menu's edge. The list takes the tap now.
• On a phone whose detector runs slower than about ten times a second, the box on a target dimmed between two results, and the controller let go of the target and the finger touched down again. The simulation put numbers on it: at 8 detections a second no box was drawn on the target for 41% of the time and the finger touched down 8 times in 10 seconds; at 5 a second, 78% and 5 times. A result that has been waited for is now taken as the newest word there is, and a target that one result missed is held, and forgotten, by how often results come. The same runs now draw a box all the time and touch down once.
• A volume key device that went away (a Bluetooth headset out of range) made the touch helper use nearly a whole processor core for as long as it ran: 182 ticks of the emulator's processor in 2 seconds, where it needs none. The helper lets the device go and says so in its log.
• The first finger put down after the touch helper took the screen over could be filed in slot 0 of the virtual touchscreen when the real one had put it in another slot, because the touchscreen does not name a slot that has not changed. A finger the emulator's touchscreen put in slot 1 showed up in slot 0 before the fix and in slot 1 after it. The helper now starts from the slot the touchscreen is at.
• The extra finger could be lifted late, or put down in the wrong order, when the touchscreen had lost events: commands that waited for the end of a group of events went out after newer ones. They go out in order once the screen has caught up.
• A touchscreen that stopped answering while the helper caught up with lost events could leave the helper reading a freed merger. It stops reading at once now.
• The touch helper says in its log when no volume key can be read, which means holding both keys will not give the touchscreen back, and it closes the key devices it could not use.
• Turning the screen left the controller remembering where it had given up on a box, and where the targets and the camera's picture had been, in the old orientation. All three are forgotten.
• A setting changed while another was being saved by AimBuddy (the learned colour, the camera it measured) could be put back to what it was, because each part took a copy of all the settings and handed it back. They change settings under the lock now, and a setting changed before the file was read is no longer lost to it.
• The detector took the GPU for being faster even if it gave a wrong answer, which some graphics drivers do (zeros, or numbers that overflowed half precision), and then saw nobody. It runs a test picture through both and uses the GPU only if its answer is the CPU's.
• A pair of model files that was imported became the model in use whatever it held, and Start then failed. The pair is loaded and run once when it is imported, and one the detector cannot read is deleted with a message, so the model in use stays as it was. Checked on the emulator with a pair of damaged files, which were refused and left nothing behind, and with a good pair, which was kept.

Upgrade note

It installs over 0.3.5-beta.1 and keeps the settings and the downloaded model. A copy of 0.3.5-beta.1 can also take it through the app's own updater, under Settings, About, Updates.

Downloads

• AimBuddy-v0.3.5-beta.2.apk is the app. Install this one.
• AimBuddy-v0.3.5-beta.2-debug.apk is a debuggable build for testing and bug reports. It installs beside the app as a separate app (com.aimbuddy.debug) and does not replace it.

Neither file contains a detection model. The app opens on the Models page, where you download one or import your own.

If Android says "App not installed as package conflicts with an existing package", the copy on your phone was signed with a different key. Uninstall it once, then install again. Releases from 0.3.0-beta.2 on are signed with the same key, so they install over each other.