Open-AutoGLM-Android

xinzezhu

1.0.3 · GitHub

Download APK

AI agents Android 7.0+ 9 months ago GPL-3.0

51 ShizuStore

4.9k GitHub

375 Stars

16 MB Size

More about this app

Automates actions on your device using the AutoGLM vision language model

Open-AutoGLM-Android

English | 中文

An Android smartphone automation assistant based on Accessibility Service, using AutoGLM vision language model to perform phone operations through natural language instructions.

Features

🤖 AI-Powered Automation: Uses AutoGLM vision language model to understand screen content and perform operations
📱 No ADB Required: Completely based on Android Accessibility Service, no computer connection needed
🎯 Natural Language Control: Describe tasks in natural language, AI automatically plans and executes
🔄 Conversational Interaction: Supports multi-step tasks, AI remembers context and continues execution
🛠️ Easy to Use: Simple interface, just input task descriptions

Interface Demo

System Requirements

Android Version: Android 11 (API 30) or higher
Device Requirements: Android device with Accessibility Service support
Network Requirements: Internet connection required to access AutoGLM API
Permission Requirements:
- Accessibility Service permission (required)
- Network access permission (required)
- Query all apps permission (for launching apps)

Usage Steps

1. Install the App

Download the APK file from the Releases page and install it on your Android device.

2. Configure API

The app comes with two pre-configured models that are automatically added to the model configuration list on first launch:

Option 1: Zhipu AutoGLM (Recommended)

Open the app, go to "Settings" page, and click "Model Configuration"
Apply for an API Key on Zhipu AI platform (visit https://open.bigmodel.cn/)
Select "智谱 AutoGLM" configuration and edit:
- API Key: Your Zhipu AI API Key
- Base URL: https://open.bigmodel.cn/api/paas/v4 (pre-configured)
- Model Name: autoglm-phone (pre-configured)
Click "Save"

Option 2: Alibaba Cloud Bailian GUI-plus

Open the app, go to "Settings" page, and click "Model Configuration"
Apply for an API Key on Alibaba Cloud Bailian platform (visit https://bailian.console.aliyun.com/)
Select "阿里云百炼 GUI-plus" configuration and edit:
- API Key: Your Alibaba Cloud DashScope API Key
- Base URL: https://dashscope.aliyuncs.com/compatible-mode/v1 (pre-configured)
- Model Name: gui-plus (pre-configured)
Click "Save"

Note: Both models support vision understanding and mobile automation. You can choose either based on your needs.

3. Enable Accessibility Service

On the settings page, click the "Go to Settings" button
Find "AutoGLM Android" in the system accessibility settings
Enable the accessibility service
Return to the app and confirm the status shows "Enabled"
In system settings, set this app's battery usage/battery policy to Unrestricted/Not restricted to prevent the system from killing background tasks

4. Start Using

Return to the "Chat" page
Enter your task description in the input box, for example:
- "Open QQ"
- "Open QQ then open xxx group"
- "Search for milk tea on Meituan"
- "Open WeChat and send a message to Zhang San"
Click the "Send" button
AI will automatically analyze the screen and perform operations
A Toast notification will be displayed when the task is completed

5. Clear Conversation

Click the delete icon on the right side of the top toolbar to clear conversation history and start a new session

Supported Operations

Launch: Launch an app
Tap: Tap screen coordinates
Type: Input text
Swipe: Swipe the screen
Back: Go back to previous page
Home: Return to home screen
Long Press: Long press
Double Tap: Double tap
Wait: Wait

Notes

Accessibility Service: The app requires accessibility service permission to work properly, please make sure it's enabled
Screenshot Function: Requires Android 11 or higher, may not work on lower versions
Emulator Limitations: Screenshot function may not work properly when running on Android emulators, it's recommended to test on real devices
Network Connection: Stable network connection is required to access AutoGLM API
API Costs: Using AutoGLM API may incur costs, please check Zhipu AI platform pricing information

FAQ

Q: Why can't I get screenshots?

A: Please ensure:

Android version is 11 (API 30) or higher
Accessibility service is enabled
If on an emulator, screenshot function may not work properly

Q: Why did the task execution fail?

A: Possible reasons:

Accessibility service is not enabled
API Key configuration is incorrect
Network connection issues
Task description is not clear enough
The system's battery optimization for this app is not set to "Unrestricted", causing background tasks to be killed

Q: Getting 4000 error when using Alibaba Cloud GUI-plus model?

A: This issue has been fixed in the latest version. The 4000 error is typically caused by invalid request parameters:

Problem: Alibaba Cloud Bailian API doesn't support the frequency_penalty parameter
Solution: The app automatically detects the model provider and removes unsupported parameters for Alibaba Cloud API
Verification: Please ensure you're using the latest version of the app; older versions may encounter this issue

If you still get a 4000 error, please check:

API Key is correct and valid
Model name is correct (should be "gui-plus")
Base URL is correctly configured as https://dashscope.aliyuncs.com/compatible-mode/v1

Q: How can I view the AI's thinking process?

A: In the chat interface, click the "Expand thinking process" button in the assistant message to view it.

Technical Architecture

UI Framework: Jetpack Compose
Architecture Pattern: MVVM (Model-View-ViewModel)
Network Requests: Retrofit + OkHttp
Data Storage: DataStore Preferences
Async Processing: Kotlin Coroutines + Flow
Accessibility Service: Android AccessibilityService API

Development

Build Requirements

Android Studio Hedgehog or higher
JDK 11 or higher
Android SDK API 30 or higher

Build Steps

# Clone the project
git clone https://github.com/xinzezhu/Open-AutoGLM-Android.git

# Open the project
cd Open-AutoGLM-Android

# Open the project in Android Studio and build

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

Thanks to the Open-AutoGLM project for inspiration
Thanks to Zhipu AI for providing the AutoGLM model

Support the Author

If this project is helpful to you, feel free to buy me a coffee ☕

Pay

Community

QQ Group: 734202636

Note: This project is for learning and research purposes only. When using this app, please comply with relevant laws and regulations and platform terms of use.

How Shizuku is used

Can grant itself system settings access via `pm grant` through Shizuku to enable its automation service silently.

This is an AI-assisted analysis of Shizuku-related usages in the app's public source code. It is best effort, so it may not catch every single usage.

How this app uses Shizuku

Shizuku is used to give the app itself system settings access so its automation can turn on without manual ADB typing.

Grant system settings access: the app runs a fixed pm grant shell command through Shizuku to give its own package WRITE_SECURE_SETTINGS, verified by checking the permission afterward.
Enable automation service silently: once it holds that access, the app writes the secure settings values for enabled accessibility services to turn on its own automation service and keep it enabled without showing the system settings screen.

Android APIs or commands used

pm grant
Runtime.getRuntime().exec
Settings.Secure.putString
Settings.Secure.putInt

Notable details

The same permission can be granted without Shizuku by running the ADB command the app copies to the clipboard. All traced Shizuku call sites use fixed commands, not user typed commands.