Script Language
既然是要寫個auto test script, 當然一樣需要有一個script language, 這邊有兩個選擇, 一是你可以選擇自己寫一個, 或是直接使用現有的語言來使用, 以現在都在講求速率的年代來說, 我們當然不會選擇前者了, 而使用現有的script language, 到底要使用什麼語言比較好咧? 比較常見的腳本語言像是Ruby, Lua, Python, 優缺點的比較有興趣的人可以看看這篇文章: Python、Lua和Ruby——脚本大P.K. , 在這裡我選擇使用了遊戲界常用的輕量化及高效能的Lua來使用。不過單純使用Lua的話, 一定沒有辦法完全符合我們的要求, 我們需要在Lua中建立新的API讓使用者可以透過它來達到與手機端溝通的功能,例如Touch, Drag, StartActivity …等各種指令, 所以我使用LuaJ這個library, 它可以很輕鬆的在Lua中新增一些API供使用者來呼叫並執行我們想要的功能, 有關LuaJ的設定在此就不多說, 只簡單講解一下在Java中新增API的方法, 假設你要在Lua中建立一個function touch(x, y):
public class Manager extends TwoArgFunction {
@Override
public LuaValue call(LuaValue modname, LuaValue env) {
LuaValue library = tableOf();
library.set("Touch", new Touch());
env.set("Manager", library);
}
public static class Touch extends TwoArgFunction {
@Override
public LuaValue call(LuaValue lv1, LuaValue lv2) {
int x = lv1.checkint();
int y = lv2.checkint();
// send touch event to device via IChimpDevice
System.out.println("Send touch event to device");
boolean ret = ActionExecutor.execute(new TouchAction(x, y));
return (ret)? LuaBoolean.TRUE:LuaBoolean.FALSE;
}
}
}完成class Manager的實作之後, 我們就可以嘗試著在Lua中呼叫實作完成的function Touch, 程式碼如下:
require 'com.around35.lua.Manager'
Manager.Touch(100, 100) // Run lua script from java
public void runLuaScript(File file) {
if ( file != null && file.exists() ) {
Globals globals = JsePlatform.standardGlobals();
LuaValue chunk = globals.loadfile(file.getAbsolutePath());
try {
chunk.call();
} catch(Exception e) {
System.out.println(e.toString());
}
}
}以下是後來新增的API:
library.set("Touch", new Touch());
library.set("Wait", new Wait());
library.set("Sleep", new Sleep());
library.set("Click", new PressImage());
library.set("Drag", new PathDrag());
library.set("Type", new Type());
library.set("PressSysKey", new PressSysKey());
library.set("Find", new ObjectFinder());
library.set("FindAll", new ObjectsFinder());
library.set("FindImage", new FindImage());
library.set("StartActivity", new StartActivity());
library.set("Capture", new SaveImage());
library.set("Shell", new Shell());
library.set("AdbPull", new AdbPull());
library.set("AdbPush", new AdbPush());圖形辨識
在上一段落所列出來的API中, 讓我們在這一個段落為大家講解一下FindImage/Click這些API的實作吧!如果我們想要點擊下面這一個dialog中的ok button, 我們要怎麼做呢?
有學過影像處理的人應該都很熟悉OpenCV這一個影像處理函式庫, 沒錯!要在一個圖像中尋找裡面的另一個sub-image,我們可以使用OpenCV中所提供的template matching, 詳細底層運作方法可以參考網址內介紹, 講解的相當完整, 下列為實作的source code:
public boolean findImage(opencv_core.IplImage src, opencv_core.IplImage template) {
opencv_core.IplImage result = matchTemplate(src, template);
double[] maxVal = new double[1];
opencv_core.CvPoint maxLoc = new opencv_core.CvPoint();
cvMinMaxLoc(result, null, maxVal, null, maxLoc, null);
cvReleaseImage(result);
return maxVal[0] >= THRESHOLD;
}
public static opencv_core.IplImage matchTemplate(opencv_core.IplImage src, opencv_core.IplImage tmp) {
opencv_core.IplImage result = cvCreateImage(
cvSize(src.width()-tmp.width()+1, src.height()-tmp.height()+1), IPL_DEPTH_32F, 1);
cvMatchTemplate(src, tmp, result, CV_TM_CCORR_NORMED);
return result;
}Java part:
public static class FindImage extends OneArgFunction {
@Override
public LuaValue call(LuaValue arg1) {
LuaString imageId = arg1.checkstring();
IplImage screen = OpenCVUtils.createFromBufferedImage(
JavaMonkey.getInstance().takeSnapshot().getBufferedImage());
IplImage template = cvLoadImage(CommonUtils.getFileName(imageId.toString()));
boolean ret = OpenCVUtils.findImage(screen, template);
screen.release();
cvReleaseImage(template);
return (ret)? LuaBoolean.TRUE:LuaBoolean.FALSE;
}
} require 'com.around35.lua.Manager'
if Manager.FindImage('ok_button') then
print('find ok button')
endJava part:
IplImage screen = OpenCVUtils.createFromBufferedImage(JavaMonkey.getInstance().takeSnapshot().getBufferedImage());
IplImage template = cvLoadImage(CommonUtils.getFileName(imageId.toString()));
int offsetX = template.width() / 2;
int offsetY = template.height() / 2;
double[] maxVal = new double[1];
CvPoint maxLoc = new CvPoint();
IplImage result = OpenCVUtils.matchTemplate(screen, template);
cvMinMaxLoc(result, null, maxVal, null, maxLoc, null);
if ( maxVal[0] >= THRESHOLD ) {
new TouchEvent(maxLoc.x() + offsetX, maxLoc.y() + offsetY).execute(device);
} require 'com.around35.lua.Manager'
Manager.Click('ok_button') require 'com.around35.lua.Manager'
function StartPAD()
Shell('am start -n jp.gungho.padHT/jp.gungho.padHT.AppDelegate')
Sleep(15.0)
end
function PressOk()
-- wait ok button for 50 seconds
Wait('ok', 50)
-- find all ok and press all of them
for i=0, 10 do
if FindImage('ok') then
Click('ok')
Sleep(5.0)
else
break
end
end
end
StartPAD()
-- wait the title 'puzzle & dragons' appears for 20 seconds
Wait('puzzle', 20)
Sleep(2.0)
-- click the screen
Touch(312,462)
Sleep(5.0)
PressOk()
if FindImage('mail') then
Click('read_mail')
end待續
沒有留言:
張貼留言