通过源码角度看看AccessibilityService
简介
创新互联建站是专业的胶州网站建设公司,胶州接单;提供成都网站设计、网站制作,网页设计,网站设计,建网站,PHP网站建设等专业做网站服务;采用PHP框架,可快速的进行胶州网站开发网页制作和功能扩展;专业做搜索引擎喜爱的网站,专业的做网站团队,希望更多企业前来合作!
AccessibilityService的设计初衷是为了辅助有身体缺陷的群体使用Android应用,它的设计贯穿着Android的控件树View, ViewGroup, ViewRootImpl体系。借助于system_server进程的中转,能够注册Accessibility事件的客户端可以具备通过system_server提供的Accessibility服务来实现监听、操作其它应用视图的功能。这个功能十分强大,可以模拟用户的行为去操作其它APP,常常被用在自动化测试、微信抢红包、自动回复等功能实现中。
写这个的初衷有二:
- 之前已经完成了Android View控件树的绘制、事件分发的源码分析,知识储备足够
- 最近接触到了一些自动化方面的项目,并且对使用无障碍服务实现的自动微信抢红包功能原理十分好奇
整体图
类图
- AccessibilityService: APP端直接继承的类,本质上是Service,通过onBind获取匿名Binder对象实现通信
- IAccessibilityServiceClientWrapper: 用于和system_server通信的匿名Binder服务
- AccessibilityInteractionClient: 本质上是个binder服务,用于获取Node信息
- AccessibilityManagerService: 运行在system_server的实名binder服务,是整体的管理类
- Service: AccessibilityManagerService的内部类,用于响应AccessibilityInteractionClient的binder通信请求
- AccessibilityInteractionConnection: 运行在被监测的APP端,提供查找、点击视图等服务
- AccessibilityManager: 运行在各个APP端,用于发送视图变化事件
- AccessibilityInteractionController: 具体视图查找、点击服务的中间控制器
- AccessibilityNodeProvider: 由客户端实现的视图节点内容提供者,最终操作的实现者
整体设计图
实例代码
public class AutoDismissService extends AccessibilityService { @Override public void onAccessibilityEvent(AccessibilityEvent event) { if (event == null) { return; } // 自动将android系统弹出的其它crash dialog取消 dismissAppErrorDialogIfExists(event); } private void dismissAppErrorDialogIfExists(AccessibilityEvent event) { // WINDOW视图变化才进行对应操作 if ((event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && event.getPackageName().equals("android")) { // 查找带有"OK"字符的可点击Node AccessibilityNodeInfo nodeInfo = findViewByText("OK", true); if (nodeInfo != null) { // 查找到后执行点击操作 performViewClick(nodeInfo); } } public AccessibilityNodeInfo findViewByText(String text, boolean clickable) { // 获取当前窗口父节点 AccessibilityNodeInfo accessibilityNodeInfo = getRootInActiveWindow(); if (accessibilityNodeInfo == null) { return null; } // 获取到满足字符要求的节点 ListnodeInfoList = accessibilityNodeInfo.findAccessibilityNodeInfosByText(text); if (nodeInfoList != null && !nodeInfoList.isEmpty()) { for (AccessibilityNodeInfo nodeInfo : nodeInfoList) { if (nodeInfo != null && (nodeInfo.isClickable() == clickable)) { return nodeInfo; } } } return null; } public void performViewClick(AccessibilityNodeInfo nodeInfo) { if (nodeInfo == null) { return; } // 由下至上进行查询,直到寻找到可点击的节点 while (nodeInfo != null) { if (nodeInfo.isClickable()) { nodeInfo.performAction(AccessibilityNodeInfo.ACTION_CLICK); break; } nodeInfo = nodeInfo.getParent(); } } }
以上是一个典型的实现Accessibility功能的JAVA代码,主要涉及三点功能:
- 当系统中有应用视图变化后,onAccessibilityEvent 方法会自动被system_server调用
- 通过AccessibilityService的getRootInActiveWindow与findAccessibilityNodeInfosByText方法,可以获取到节点信息
- 通过AccessibilityNodeInfo的performAction方法,最终会在被监听APP中执行对应操作
本篇文章将会围绕着这三点主要功能进行源码分析
源码分析
常见 AccessibilityEvent 事件种类
序号 | 种类名称 | 触发时机 |
---|---|---|
1 | TYPE_VIEW_CLICKED | 可点击的组件被点击 |
2 | TYPE_VIEW_LONG_CLICKED | 可点击的组件被长按 |
3 | TYPE_VIEW_SELECTED | 组件被选中 |
4 | TYPE_VIEW_FOCUSED | 组件获取到了焦点 |
5 | TYPE_VIEW_TEXT_CHANGED | 组件中的文本发生变化 |
6 | TYPE_VIEW_SCROLLED | 组件被滑动 |
7 | TYPE_WINDOW_STATE_CHANGED | dialog等被打开 |
8 | TYPE_NOTIFICATION_STATE_CHANGED | 通知弹出 |
9 | TYPE_WINDOW_CONTENT_CHANGED | 组件树发生了变化 |
onAccessibilityEvent 触发流程
这里以TextView.setText触发事件变化流程为例进行分析
TextView.setText
应用组件状态发生变化
frameworks/base/core/java/android/widget/TextView.java
private void setText(CharSequence text, BufferType type, boolean notifyBefore, int oldlen) { ... notifyViewAccessibilityStateChangedIfNeeded(AccessibilityEvent.CONTENT_CHANGE_TYPE_TEXT); ... } public void notifyViewAccessibilityStateChangedIfNeeded(int changeType) { if (!AccessibilityManager.getInstance(mContext).isEnabled() || mAttachInfo == null) { return; } if (mSendViewStateChangedAccessibilityEvent == null) { // 本质上是一个Runnable,意味着这里的流程会进入异步处理 mSendViewStateChangedAccessibilityEvent = new SendViewStateChangedAccessibilityEvent(); } mSendViewStateChangedAccessibilityEvent.runOrPost(changeType); } private class SendViewStateChangedAccessibilityEvent implements Runnable { ... @Override public void run() { mPosted = false; mPostedWithDelay = false; mLastEventTimeMillis = SystemClock.uptimeMillis(); if (AccessibilityManager.getInstance(mContext).isEnabled()) { final AccessibilityEvent event = AccessibilityEvent.obtain(); event.setEventType(AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED); event.setContentChangeTypes(mChangeTypes); // 最终TYPE_WINDOW_CONTENT_CHANGED事件在这里异步发送 sendAccessibilityEventUnchecked(event); } mChangeTypes = 0; } ... } public void sendAccessibilityEventUnchecked(AccessibilityEvent event) { if (mAccessibilityDelegate != null) { mAccessibilityDelegate.sendAccessibilityEventUnchecked(this, event); } else { sendAccessibilityEventUncheckedInternal(event); } } public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) { host.sendAccessibilityEventUncheckedInternal(event); } public void sendAccessibilityEventUncheckedInternal(AccessibilityEvent event) { if (!isShown()) { return; } ... // 此处交由TextView所在父View进行处理,为责任链模式,事件经过层层向上传递,最终交由ViewRootImpl进行处理 ViewParent parent = getParent(); if (parent != null) { getParent().requestSendAccessibilityEvent(this, event); } }
ViewRootImpl.requestSendAccessibilityEvent
ViewRootImpl将事件派发到system_server
frameworks/base/core/java/android/view/ViewRootImpl.java
@Override public boolean requestSendAccessibilityEvent(View child, AccessibilityEvent event) { ... // 本地调用到AccessibilityManager进行事件发送 mAccessibilityManager.sendAccessibilityEvent(event); return true; }
frameworks/base/core/java/android/view/accessibility/AccessibilityManager.java
public void sendAccessibilityEvent(AccessibilityEvent event) { final IAccessibilityManager service; final int userId; synchronized (mLock) { // 获取system_server的Accessibility实名服务 service = getServiceLocked(); ... } try { ... long identityToken = Binder.clearCallingIdentity(); // binder call 到服务端,进行事件分发中转 doRecycle = service.sendAccessibilityEvent(event, userId); Binder.restoreCallingIdentity(identityToken); ... } catch (RemoteException re) { Log.e(LOG_TAG, "Error during sending " + event + " ", re); } finally { ... } }
AccessibilityManagerService.sendAccessibilityEvent
system_server将事件分发到各个监听组件变化的Service
frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java
// binder call 到服务端,触发事件派发 @Override public boolean sendAccessibilityEvent(AccessibilityEvent event, int userId) { synchronized (mLock) { ... if (mSecurityPolicy.canDispatchAccessibilityEventLocked(event)) { ... notifyAccessibilityServicesDelayedLocked(event, false); notifyAccessibilityServicesDelayedLocked(event, true); } ... } return (OWN_PROCESS_ID != Binder.getCallingPid()); } private void notifyAccessibilityServicesDelayedLocked(AccessibilityEvent event, boolean isDefault) { try { UserState state = getCurrentUserStateLocked(); for (int i = 0, count = state.mBoundServices.size(); i < count; i++) { Service service = state.mBoundServices.get(i); if (service.mIsDefault == isDefault) { if (canDispatchEventToServiceLocked(service, event)) { // 调用内部服务,以触发事件派发 service.notifyAccessibilityEvent(event); } } } } catch (IndexOutOfBoundsException oobe) { ... } } class Service extends IAccessibilityServiceConnection.Stub implements ServiceConnection, DeathRecipient { public void notifyAccessibilityEvent(AccessibilityEvent event) { synchronized (mLock) { ... if ((mNotificationTimeout > 0) && (eventType != AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED)) { ... // 按照惯例,异步分发到客户端进行派发 message = mEventDispatchHandler.obtainMessage(eventType); } else { message = mEventDispatchHandler.obtainMessage(eventType, newEvent); } mEventDispatchHandler.sendMessageDelayed(message, mNotificationTimeout); } } } public Handler mEventDispatchHandler = new Handler(mMainHandler.getLooper()) { @Override public void handleMessage(Message message) { final int eventType = message.what; AccessibilityEvent event = (AccessibilityEvent) message.obj; notifyAccessibilityEventInternal(eventType, event); } }; private void notifyAccessibilityEventInternal(int eventType, AccessibilityEvent event) { IAccessibilityServiceClient listener; ... // mServiceInterface是通过bind客户端的AccessibilityService,在onServiceConnected连接成功后,获取到binder proxy转化来的,以这种方式实现了system_server与客户端的通信 listener = mServiceInterface; ... try { listener.onAccessibilityEvent(event); if (DEBUG) { Slog.i(LOG_TAG, "Event " + event + " sent to " + listener); } } catch (RemoteException re) { Slog.e(LOG_TAG, "Error during sending " + event + " to " + listener, re); } finally { event.recycle(); } }
AccessibilityService.onAccessibilityEvent
APP接收到组件变化的事件,并可以选择做出相应的处理
frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java
// 抽象方法,模板模式,被系统主动调用 public abstract void onAccessibilityEvent(AccessibilityEvent event); // 该service是被system_server主动绑定的,获取到IAccessibilityServiceClientWrapper的proxy来实现系统的主动调用 @Override public final IBinder onBind(Intent intent) { return new IAccessibilityServiceClientWrapper(this, getMainLooper(), new Callbacks() { ... @Override public void onAccessibilityEvent(AccessibilityEvent event) { AccessibilityService.this.onAccessibilityEvent(event); } ... } } // 收到binder调用后,使用handler异步进行事件的处理 public void onAccessibilityEvent(AccessibilityEvent event) { Message message = mCaller.obtainMessageO(DO_ON_ACCESSIBILITY_EVENT, event); mCaller.sendMessage(message); } @Override public void executeMessage(Message message) { switch (message.what) { case DO_ON_ACCESSIBILITY_EVENT: { AccessibilityEvent event = (AccessibilityEvent) message.obj; if (event != null) { AccessibilityInteractionClient.getInstance().onAccessibilityEvent(event); // 通过回调调用以触发事件 mCallback.onAccessibilityEvent(event); ... } } return; } }
getRootInActiveWindow 父节点获取流程
在调用findAccessibilityNodeInfosByText之前,需要通过getRootInActiveWindow方法获取到父节点,才能通过调用父AccessibilityNodeInfo的方法进行其子节点信息查询
AccessibilityService.getRootInActiveWindow
frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java
public AccessibilityNodeInfo getRootInActiveWindow() { // 查找父节点的操作没有在自己的类中实现,而是交由了同一进程的Client管理类进行处理 return AccessibilityInteractionClient.getInstance().getRootInActiveWindow(mConnectionId); }
frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java
public AccessibilityNodeInfo getRootInActiveWindow(int connectionId) { return findAccessibilityNodeInfoByAccessibilityId(connectionId, AccessibilityNodeInfo.ACTIVE_WINDOW_ID, AccessibilityNodeInfo.ROOT_NODE_ID, false, AccessibilityNodeInfo.FLAG_PREFETCH_DESCENDANTS); } public AccessibilityNodeInfo findAccessibilityNodeInfoByAccessibilityId(int connectionId, int accessibilityWindowId, long accessibilityNodeId, boolean bypassCache, int prefetchFlags) { ... // 尝试binder call到system_server,请求中转到其它APP进程中查询父节点信息,注意的是这里AccessibilityInteractionClient本身是个binder服务端,把this传到system_server后,其它进程可以通过这个引用拿到binder proxy,以实现通信 final boolean success = connection.findAccessibilityNodeInfoByAccessibilityId( accessibilityWindowId, accessibilityNodeId, interactionId, this, prefetchFlags, Thread.currentThread().getId()); Binder.restoreCallingIdentity(identityToken); // If the scale is zero the call has failed. if (success) { // 调用成功后,这里会尝试同步获取结果 Listinfos = getFindAccessibilityNodeInfosResultAndClear( interactionId); finalizeAndCacheAccessibilityNodeInfos(infos, connectionId); if (infos != null && !infos.isEmpty()) { return infos.get(0); } } ... }
Service.findAccessibilityNodeInfoByAccessibilityId
注意一下,这里的Service不是Android中的四大组件的Service,取名叫AccessiblitManagerServiceInternal其实更合适
frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java
@Override public boolean findAccessibilityNodeInfoByAccessibilityId( int accessibilityWindowId, long accessibilityNodeId, int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags, long interrogatingTid) throws RemoteException { ... // 获取到其他APP的节点获取服务 IAccessibilityInteractionConnection connection = null; ... resolvedWindowId = resolveAccessibilityWindowIdLocked(accessibilityWindowId); ... if (!permissionGranted) { return false; } else { connection = getConnectionLocked(resolvedWindowId); if (connection == null) { return false; } } ... // 这里的callback为之前应用的服务proxy句柄,将它传入是为了之后的信息通信不再需要经过system_server中转,而是直接可以APP对APP的进行通信 connection.findAccessibilityNodeInfoByAccessibilityId(accessibilityNodeId, partialInteractiveRegion, interactionId, callback, mFetchFlags | flags, interrogatingPid, interrogatingTid, spec); ... }
AccessibilityInteractionConnection.findAccessibilityNodeInfoByAccessibilityId
这里调用到了APP端,其实同onAccessibilityEvent调用流程一样,是APP->SYSTEM->APP的调用顺序
frameworks/base/core/java/android/view/ViewRootImpl.java
@Override public void findAccessibilityNodeInfoByAccessibilityId(long accessibilityNodeId, Region interactiveRegion, int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags, int interrogatingPid, long interrogatingTid, MagnificationSpec spec) { ViewRootImpl viewRootImpl = mViewRootImpl.get(); if (viewRootImpl != null && viewRootImpl.mView != null) { // 这里也只是委托给控制类进行细节操作的处理 viewRootImpl.getAccessibilityInteractionController() .findAccessibilityNodeInfoByAccessibilityIdClientThread(accessibilityNodeId, interactiveRegion, interactionId, callback, flags, interrogatingPid, interrogatingTid, spec); } else { ... } }
frameworks/base/core/java/android/view/AccessibilityInteractionController.java
private void findAccessibilityNodeInfoByAccessibilityIdUiThread(Message message) { ... // 初始化将会返回的节点 Listinfos = mTempAccessibilityNodeInfoList; infos.clear(); try { if (mViewRootImpl.mView == null || mViewRootImpl.mAttachInfo == null) { return; } mViewRootImpl.mAttachInfo.mAccessibilityFetchFlags = flags; View root = null; if (accessibilityViewId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { root = mViewRootImpl.mView; } else { root = findViewByAccessibilityId(accessibilityViewId); } ... } finally { try { ... adjustIsVisibleToUserIfNeeded(infos, interactiveRegion); // 通过callback binder proxy句柄,将节点信息binder回应用 callback.setFindAccessibilityNodeInfosResult(infos, interactionId); infos.clear(); } catch (RemoteException re) { /* ignore - the other side will time out */ } ... } }
AccessibilityInteractionClient.setFindAccessibilityNodeInfosResult
frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java
public void setFindAccessibilityNodeInfosResult(Listinfos, int interactionId) { synchronized (mInstanceLock) { if (interactionId > mInteractionId) { if (infos != null) { ... // 设置应用的返回节点信息 if (!isIpcCall) { mFindAccessibilityNodeInfosResult = new ArrayList<>(infos); } else { mFindAccessibilityNodeInfosResult = infos; } } else { mFindAccessibilityNodeInfosResult = Collections.emptyList(); } mInteractionId = interactionId; } // 释放锁,停止等待,节点信息已经取回 mInstanceLock.notifyAll(); } }
findAccessibilityNodeInfosByText与performAction 对目标节点进行操作
AccessibilityNodeInfo.findAccessibilityNodeInfosByText
找到父节点信息后,就可以通过父节点获取对应的子节点信息了
frameworks/base/core/java/android/view/accessibility/AccessibilityNodeInfo.java
public ListfindAccessibilityNodeInfosByText(String text) { ... // 一样的流程,通过AccessibilityInteractionClient去获取信息 AccessibilityInteractionClient client = AccessibilityInteractionClient.getInstance(); return client.findAccessibilityNodeInfosByText(mConnectionId, mWindowId, mSourceNodeId, text); } ```
以下的代码流程同getRootInActiveWindow大概一致,就不详细分析了
#### AccessibilityNodeInfo.performAction
获取到对应子节点后,通过performAction可以执行对应的操作了,如常用的点击
最终回调用到AccessibilityInteractionController,获取到AccessibilityProvier后就可以执行performAction的最终操作了
frameworks/base/core/java/android/view/AccessibilityInteractionController.java
```java private void performAccessibilityActionUiThread(Message message) { View target = null; if (accessibilityViewId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { target = findViewByAccessibilityId(accessibilityViewId); } else { target = mViewRootImpl.mView; } if (target != null && isShown(target)) { AccessibilityNodeProvider provider = target.getAccessibilityNodeProvider(); if (provider != null) { if (virtualDescendantId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { // 在客户端执行performAction操作 succeeded = provider.performAction(virtualDescendantId, action, arguments); } else { succeeded = provider.performAction(AccessibilityNodeProvider.HOST_VIEW_ID, action, arguments); } } else if (virtualDescendantId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { succeeded = target.performAccessibilityAction(action, arguments); } } }
frameworks/base/core/java/android/view/View.java
public boolean performAccessibilityActionInternal(int action, Bundle arguments) { ... switch (action) { case AccessibilityNodeInfo.ACTION_CLICK: { if (isClickable()) { // 最终调用到我们熟悉的View.performClick方法 performClick(); return true; } } break; ... }
分析到这里可以看到,Accessibility服务框架类似于hook在Android View组件树中的一套实现,它并不是独立的一套机制,而是”寄生”在View的显示、事件分发的流程中。
总结
功能实现依赖于ViewRootImpl, ViewGroup, View视图层级管理的基本架构。在视图变化时发出事件、当收到视图操作请求时也能够作出响应。
system_server在实现该功能的过程中扮演着中间人的角色。当被监听APP视图变化时,APP首先会发出事件到system_server,随后再中转到监听者APP端。当监听者APP想要执行视图操作时,也是首先在system_server中找到对应的客户端binder proxy,再调用相应接口调用到被监听APP中。完成相关操作后,通过已经获取到的监听APP binder proxy句柄,直接binder call到对应的监听客户端。
无障碍权限十分重要,切记不可滥用,APP自身也需要有足够的安全意识,防止恶意应用通过该服务获取用户隐私信息
好了,以上就是这篇文章的全部内容了,希望本文的内容对大家的学习或者工作具有一定的参考学习价值,如果有疑问大家可以留言交流,谢谢大家对创新互联的支持。
网站题目:通过源码角度看看AccessibilityService
网站地址:http://pwwzsj.com/article/jehiei.html