piątek, 16 grudnia 2011

Cracow.mobi review

Last week, I visited Cracow and its first edition of conference on mobile application developement.
I was delivering a presentation about Testing in Android (in Polish). It was my second try with this presentation.
The first time was during mobilization.pl conference in Łódź, which I also coorganized last October.

The event was held in conference centre of Technical University of Kraków: Kotłownia (boiling room) and GIL .
Registration was a bit crowdy but in a minute I got nice set of Android stickers and id. Some students came to me and said they liked the presentation in mobilization.pl despite the fact that examples (maven!) did not work.

Coffee was served downstairs but the place was not labelled properly I think so not many people knew it.
There was no food provided for atendees but I was stressed with my presentation so I did not care so much.

First day of the conference had two tracks (Android and iOS) held concurrently. I stayed in Android because my presentation was the 3rd.
The Kotłownia site was really great. The screen was big, there was also LCD screen for the presenter so speaker could see his slides without turning back on the attendees.
What is more the room had 2nd level being also equiped with a small screen and few rows of chairs for attendees.

The organizers already announced some changes in the agenda due to abscence of speaker (in iOS track) and then conference started.
The conference started strong. First Android speaker was Pawel Zieba and talked about sqlite. I liked the technical content he delivered with some minor exceptions to some presentation skills (sometimes too quiet). Pawel got lot of question from the public so it means it was interesting for many.
Next presentation was eagerly awaited by me. Konrad Malawski was talking about roboguice. I know what roboguice is, I even once aimed at presenting about it but never actually got into. Konrad made really good impression on me with his presentation. Roboguice seem to make code look better. The presenter answered smoothly real-life questions so I really believe he is using not only gives presentations about it.

I will not judge my own presentation. I am still not very happy about it, maybe next conference it will go smoothly. Again I had some problems with examples and multiscreens.
I need consider showing videos instead and not switching to eclipse.

After lunch there was a presentation from Jacek Laskowski who just started learning Android, half a year ago. Jacek is a person who inspires me and many people from JUGs in Poland. It was a pleasure to see Jacek presentation skills (great voice, good interaction with audience) and I think he made the most of few lines of code he presented.

Next presentation by applicake guy was about content synchronization. Unfortunately I haven't seen that fully so I am not able to judge it.

Next day seemed to be less rich in presentations for me but started strong with very inspiring speech by Łukasz Mikulski and games in our life.
Cool transitions (I guess thanks to Mac's keynote), embedded videos and real life examples made it good start of the day.
This speaker was late so before he started there was a really cool round of "introduce yourself" with everybody having a microphone to say few sentences about themselves. There was a bunch of interesting people representing startups or portals.
The agenda was already a mess so noone know at what time next presentation starts.

I moved to GIL to see the presentation I missed during mobilization.pl from Bartłomiej Zass from Microsoft. He is proffessional speaker and he confirmed that: technical problems were solved with grace, good interaction with attendees, no difficult questions. After lunch I come back to GIL to see presentations about phonegap. Krzysztof "Procek" Ścira was natural born speaker. It was smooth and he talked about practise giving many useful tips for people who like to start the adventure with phonegap. His presentation attracted only handful of people (due to problems with agenda I guess) and he treated this group with great respect and did very good presentation with focus on practical things (project setup etc.).

To sum up, I think that event was good but there is lot of to improve for the next edition.
I liked:
+ general idea :)
+ android stickers and t-shirt (I got 3 :) - as a speaker, for answering questions and for being mobiledeveloper.pl fan :))
+ meeting few people I knew before only from twitter
+ having a chance to see Jacek Laskowski in action and spare few words with him

I did not liked:
- agenda changes, moreover poorly advertised
- speakers that did not arrive (yes I know is not the fault of organizers, we faced the same during mobilization.pl)
- afterparty on the last day, should be on Friday

niedziela, 24 kwietnia 2011

Converting release keys to debug

Tests in android are like applications. They are organized into apks. They are installed along with application under test.
For security reasons they both require to be signed by the same keystore.

If you use different key for the application and the test you will get error:

Test run failed: Permission Denial: starting instrumentation ComponentInfo{net.retsat1.starlab.test/android.test.InstrumentationTestRunner} from pid=305, uid=305 not allowed because package net.retsat1.starlab.test does not have a signature matching the target net.retsat1.starlabjava.lang.SecurityException: Permission Denial: starting instrumentation ComponentInfo{net.retsat1.starlab.test/android.test.InstrumentationTestRunner} from pid=305, uid=305 not allowed because package net.retsat1.starlab.test does not have a signature matching the target net.retsat1.starlab


During development cycle keys generated by eclipse are usually used. When you are starting to think about release you are preparing the build signed with release key.

In eclipse there is no automatic way to sign for release, so you probably hand over this to ant script or do the signing by hand.

Afterwards if you are lucky your test that had previously run successfully on debug mode will be still passing in release mode. If for some reason this is not true you want to debug your test with eclipse.

When you want to run test(s) against your new build you find that signatures of the test and release apk mismatch.

In eclipse you can set a custom keystore from menu

Windows -> Preferences -> Android - > Build


This unfortunately leads to a problem as you eclipse does not accept release keystore with an error: Keystore was tampered with, or password was incorrect.

To convert release key to debug one that will be acceptable for eclipse, you need to import keystore into new debug keystore with keytool from jdk. This is done with the following command.

keytool -importkeystore -v -srckeystore release.keystore -destkeystore custom-debug.keystore -srcstorepass release-pass -deststorepass android -srcalias release-key -destalias androiddebugkey -srckeypass release-pass -destkeypass android


Of course substitute passwords and alias of source keystore with the one used in creation of release keystore.

wtorek, 29 marca 2011

Taking a screenshot in Android

There are plenty quite few decent methods for taking a screenshot in Android.

  • Monkey runner that comes with sdk 2.3 (works in 2.2 (checked) and earlier (assumed))
    http://developer.android.com/guide/developing/tools/monkeyrunner_concepts.html#SampleProgram
    This is pretty cool, python thing. I just wonder if that can be used within android with Android Scripting Engine. Definitely good choice for automated things like tests.
  • Reading framebuffer with an adb command
    adb pull /dev/graphics/fb0 screenshot.raw

    and processing this with some geeky commands. Might be impressive for your geekfriends but is not very simple to use.
    More about it you can read here here and there. The problem is that you have to count the buffer size, know the format of framebuffer and is not quite obvious.
  • Using android screenshot library maintained by polidea
  • Using ddms tool from sdk

    Pretty simple but you can't use it in an environment in automated, operatorless (no human involved) way.
  • Using a hierarchy viewer tool. With that you can take screenshots not only of the whole screen but also individual controls (views in android jargon).
  • Writing something around ddmlib. In fact all the tools above except ASL use ddmlib.
  • Last, but definetely not least is something by google itself. No surprise done on top of ddmlib.
    This tool is called screenshot and is in java. You can find that in sdk
    sdk/screenshot/src/com/android/screenshot/Screenshot.java.
    I am pretty sure you can compile it standalone with ddmlib as only dependency, but I had already built my sdk and it was compiled in

    android-source/out/host/linux-x86/bin/screenshot2
    and that simply works!
    ./screenshot2
    Usage: screenshot2 [-d | -e | -s SERIAL] [-l] OUT_FILE
    -d Uses the first device found.
    -e Uses the first emulator found.
    -s Targets the device by serial number.
    -l Rotate images for landscape mode.


Enjoy!

czwartek, 10 marca 2011

Mavenize existing android eclipse project

I have been working some time on android projects that try to use maven. We grew with the maven android plugin and maven/android eclipse plugin.
In this article I wanted to create guide how to mavenize existing android/eclipse project. During writing this post I discovered that few things have been different with new plugins (android or eclipse) and changed for better.

I used git to track what changes are being done by each steps described here to show the diff(s) and explain better what has happened.


  1. Create eclipse project for transition

    The project to show the mavenization process has been created with following parameters.

    Application name: MavenizeMe
    Package: net.retsat1.starlab.mavenize
    Activity: Me
    Sdk: 8

  2. Create android maven project from archetype
    https://github.com/akquinet/android-archetypes/wiki/android-quickstart-archetype

    mvn archetype:generate \
    -DarchetypeArtifactId=android-quickstart \
    -DarchetypeGroupId=de.akquinet.android.archetypes \
    -DarchetypeVersion=1.0.5 \
    -DgroupId=net.retsat1.starlab.mavenize \
    -DartifactId=mavenize-me

    To change sdk to 8 you need to answer 'N' for the first question and correct the default 7.

  3. Copy pom.xml to the project with android

  4. Import the project as maven project.

    When the project gets imported you will notice changes in .project and .classpath.

    .project shows that the project has new nature of maven nature.
    One of the builders is removed and instead we have maven builder.

                     </arguments>  
    </buildCommand>
    <buildCommand>
    - <name>com.android.ide.eclipse.adt.ApkBuilder</name>
    + <name>org.maven.ide.eclipse.maven2Builder</name>
    <arguments>
    </arguments>
    </buildCommand>
    </buildSpec>
    <natures>
    + <nature>org.maven.ide.eclipse.maven2Nature</nature>
    <nature>com.android.ide.eclipse.adt.AndroidNature</nature>
    <nature>org.eclipse.jdt.core.javanature</nature>
    </natures>

    I have been having lot of problems when apk builder was enabled. It did not go well with maven builder and I was getting some random errors during the build. I think the earlier versions of the plugin had not been removing apk builder and it clashed with maven very often.

    .classpath has a bunch of new entries
     -     <classpathentry kind="src" path="src"/>  
    - <classpathentry kind="src" path="gen"/>
    + <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/J2SE-1.5"/>
    + <classpathentry kind="con" path="org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER"/>
    <classpathentry kind="con" path="com.android.ide.eclipse.adt.ANDROID_FRAMEWORK"/>
    - <classpathentry kind="output" path="bin"/>
    + <classpathentry kind="src" output="target/android-classes" path="gen">
    + <attributes>
    + <attribute name="optional" value="true"/>
    + </attributes>
    + </classpathentry>
    + <classpathentry kind="output" path="target/android-classes"/>


    Not sure why src has been removed. I reverted it back by rightclick on src and add to build path.

    There is a wide belief that eclipse files should not be committed to version control but should be generated for every developer on his own machine. In general I believe that is true but for android maven project this might be exception to the rule.
    In this case these files are already there :) What is more, mvn eclipse:eclipse generates something that does not work well as in case of regular java projects.

  5. Enabling ADT to run and debug.

    After cleaning the project I noticed that the project cannot be run with eclipse adt plugin. Could not find mavenize-me.apk!

    Maven does produce apk, but it has version number in the name./target/mavenize-me-1.0-SNAPSHOT.apk.

    At the same time ADT plugin expects the apk with the name as the project at the output folder.

    The solution is to either force maven to generate a project without a version information or copy generated apk to the location that adt expects. I used the second solution with the help of maven-ant-plugin.
     <plugin>  
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.6</version>
    <executions>
    <execution>
    <phase>package</phase>
    <configuration>
    <target>
    <property name="source"
    value="${project.build.directory}/${project.artifactId}-${project.version}.${project.packaging}" />
    <property name="destination"
    value="${project.build.directory}/classes/${project.name}.apk" />
    <delete file="${destination}" />
    <copy file="${source}" tofile="${destination}" />
    </target>
    </configuration>
    <goals>
    <goal>run</goal>
    </goals>
    </execution>
    </executions>
    </plugin>


  6. Fixing the source location

    The apk generated by maven does not contain anything except the resource so the process crashes on emulator. It does not see the activity class Me. The default maven for java code is src/main/java but adt plugin prefers the src directory.

    We can either setup maven to work with
     <sourceDirectory>src</sourceDirectory>  
    or move the src folder to the location that maven prefers to work with. I chose the latter.

    mkdir -p src/main/java
    git mv src/net/ src/main/java/

    And set the eclipse classpath to new location. First you need to remove src from the path and add it src/main/java
    .classpath will show the diff as follows
     -    <classpathentry kind="src" path="src"/>  
    + <classpathentry kind="src" path="src/main/java"/>


  7. Remove the builders

    I did not understand why we need other builders than maven and java so I removed ResourceBuilder and Precompiler from the .project.

         </projects>  
    <buildSpec>
    <buildCommand>
    - <name>com.android.ide.eclipse.adt.ResourceManagerBuilder</name>
    - <arguments>
    - </arguments>
    - </buildCommand>
    - <buildCommand>
    - <name>com.android.ide.eclipse.adt.PreCompilerBuilder</name>
    - <arguments>
    - </arguments>
    - </buildCommand>
    - <buildCommand>
    <name>org.eclipse.jdt.core.javabuilder</name>
    <arguments>
    </arguments>


    It turned out when I added new id to the main.xml that it is not visible in net.retsat1.starlab.mavenize.R. The eclipse project does not compile and as a result run button does not work. Also the project reports errors. Maven on the other hand successfully compiles the project.

    The id is generated in
    target/generated-sources/r/net/retsat1/starlab/mavenize but not in gen folder by adt. No wonder, because I removed the builder responsible for that. But if it is generated in the r directory we do not need gen folder. It must be on the build path. Deleting gen will cause adt to complain about missing problem so I needed a nifty way to deceive eclipse. This way is to create a linked folder name gen pointing to the r folder. We use eclipse variable for that


    PROJECT_LOC/target/generated-sources/r


     +        <link>  
    + <name>gen</name>
    + <type>2</type>
    + <locationURI>PROJECT_LOC/target/generated-sources/r</locationURI>
    + </link>



  8. The maven way

    At this moment I had source in the standard maven but still assets and res folders at root that does not look like a standard maven. I am not sure where would be the correct place in maven for them but chose src/main/android-assets and src/main/android-res. To make adt happy I did the same trick as with link to generated sources.

     +        <link>  
    + <name>assets</name>
    + <type>2</type>
    + <locationURI>PROJECT_LOC/src/main/android-assets</locationURI>
    + </link>
    + <link>
    + <name>res</name>
    + <type>2</type>
    + <locationURI>PROJECT_LOC/src/main/android-res</locationURI>
    + </link>


    The defaults in maven-android-plugin are the same as for adt plugin so is necessary to adjust those paths.

     -                    <assetsDirectory>${project.basedir}/assets</assetsDirectory>  
    - <resourceDirectory>${project.basedir}/res</resourceDirectory>
    + <assetsDirectory>${project.basedir}/src/main/android-assets</assetsDirectory>
    + <resourceDirectory>${project.basedir}/src/main/android-r</resourceDirectory>


  9. The maven project

    In those steps I achieved the project which is compiled solely by maven. The project is also using adt plugin for running and debugging so creates similar experience for the developers accustomed to adt.

    I am sure this is rather lengthy operation to convert to maven project and I had not discoverd many problems that bigger android maven projects may have on such trivial example. I have not considered a project with android tests.

    Another drawback of this setup is that Maven as a project builder is not very convenient. Is much slower than adt and also does not support incremental builds perfectly. In the background maven compiler is invoked constantly eventough there are no changes ins the source code. That might be little annoying.

    The pros for such setup is for sure the dependency management that maven may give you.
    On top of that also your continuous integration system will be more than happy to receive maven project and should generate the same artifact(s) as on developers workstations. I do not like when developers use different method of building the projects(on their workstation) than the official build does. Is often the case of works for me only or send my build to the customer scenario.

środa, 9 marca 2011

Getting started with Android & Maven

Is hard to imagine a project that is expected to deliver something without having a build script that automates the process. Batch/shell script is a primitive example but ant and maven are well established industry standards for build solutions. Maven scope is slightly bigger than that but main part of maven is for building.

Android folks have selected ant as their choice. In the sdk you can find ant examples (platforms/android-8/ant) and by using android create project command you can get a skeleton of android project that is built by ant.

Maven is widely considered as next step in evolution after ant so no wonder that many people start asking question if there is a support for android in maven.

Android maven plugin is the answer they look for. To get started there are few ways to do it

When analyzing samples, at the first glance you will notice that standard maven directory layout is not preserved. The reason for that is that maven android plugin does not want to go into the way of android development tools (adt) layout. People accustomed to maven will find it slightly odd.
To work with eclipse you obviously need m2eclipse plugin and maven-android-integration plugin. You can find that with eclipse marketplace directly from eclipse.
After plugins are installed the next thing you do is to import the project in eclipse (Import -> Existing maven project), it does not work on start but after clean or two you have a compilable maven + android project.

When project is imported in eclipse it will be treated as maven project but also will have android nature. This allows the developer to use a Run As Android Application (and Debug).


When you go to a builders configuration page (Projects -> Properties -> Builders) you should see 4 builders, 2 from adt (android resource builder and precompiler) and java and maven builder. I don't think that the 2 former are really necessary for the project. At least disabling them does not cause anything noticeably bad.

To learn what is available from the android plugin use help:describe goal.
mvn -Dplugin=android help:describe -Dfull=true This would display a lengthy list with the targets you may use along with the configuration xmls that can be placed in the pom.xml in plugin configuration.

Most important goals are
  • android:apk - to create android pakcage
  • android:deploy - to deploy on emulator/device
  • android:emulator-start

My own opinion



I must admit I was never a fan of maven. Mainly because mobile projects I was involved in where rather small, having no dependencies and mostly ant based. When I tried learn on my own I was put off by the conventions imposed by maven. My prejudice towards maven would last probably longer hadn't been I forced to use maven. Then I gradually discovered the power of auto dependency management and the benefits of conventions.


As android is concerned I still find some clashes but the maven plugin is strongly developed. Mailing list is active, development is active and new features arrive at good pace (recently apklib).
Still have not dwelt very deeply in android maven development so I cannot say is industry ready yet or not but the future looks promising.

środa, 9 lutego 2011

Running a headless emulator

Knowing that headless system is a system designed to be run without a monitor you might
wonder if this really makes sense to have an emulator running without a possibility to have a look what is going on there :).

That might sound crazy but think of the continuous integration like hudson. Hudson will run the test(s) to check the stability of the build and generates reports afterwards. No one will be interested in sitting in front of the screen connected to hudson to check if the test are running ok and when the test fails. This can be obtained from the reports or the test logs and repeated (hopefully) on developers machines with the debugger connected.

Also when the UI test fails they will be discarded from the screen so you might not be even able to spot where the failure occurred.

Finally to run emulator you need X server running, a DISPLAY variable exported so might be too much of setup for simple continuous integration server.

In order to run emulator in headless mode issue a command

emulator @Android2.2 -no-window

And that is explained in help window

emulator -help
...
-no-window disable graphical window display

wtorek, 8 lutego 2011

Randomizing the order of the tests

One of JUnit best practices say that one should not assume the order in which tests in a test case are run. It is assumed that method will be executed in arbitrary order.
Therefore the having dependencies between the tests is discouraged.

JUnit uses reflection to find the test cases and it is not guaranteed that the order will be the same in different VM(s). But, you do not change your vm so often so the order of the test remains usually persistent across the lifetime of the project.

I have seen the tests that people do make some assumptions about the order and one test "prepares" the data or environment for the next. And the result is that test may fail if executed separately.

The dependencies might not be intentional at all, and just be the result of the way the test were ordered and no one realizes that they exist.

In the project I am right now we wanted to ensure that there are no dependencies between the tests and the solution was to intentionally make the order unpredictable. The problem was to find a way to randomize the test cases.

In android (2.2) source code when you look at
android.test.InstrumentationTestRunner.onCreate(Bundle)
you will see that
android.test.suitebuilder.TestSuiteBuilder.build()
is using something called a test grouping that is constructed with
android.test.suitebuilder.TestGrouping.SORT_BY_FULLY_QUALIFIED_NAME
and later adds a test(s) from a testCases list.

I am not sure in which order is this list populated but I am guessing the order can be predicted. As a result we will get a almost constant sequence of the test(s) unless some new test(s) are introduced.

So looking at the method android.test.InstrumentationTestRunner.onCreate(Bundle)
showed that the set of test is created inside the method using a local instance of TestSuiteBuilder and is assigned to the AndroidTestRunner instance. A class instance but the getter getAndroidTestRunner() everytime return new instance of the runner.

Recently I became more comfortable (& skilled) in java reflection and at first thought I expected to be able to get to that field with reflection. The problem was that onCreate just starts the test just before it completes. So you might not be able to randomize or you will randomize when the test are executed.

What is more with reflection you will have an ugly code, with lot of exception catching and without compiler checks so error prone and hard to maintain. Consequently, is very fragile and if the field name will change it will fail badly. Last but not least is a common belief the reflection is very slow and even though I had not measured it I share the same opinion.

The reflection might be used between onCreate and onStart but I figured out that we can save the runner used in onCreate and in onStart or start shuffle the test(s).

The result is below
 package net.retsat1.starlab.testrandomizer;  
import java.util.Collections;
import java.util.List;
import junit.framework.TestCase;
import android.test.AndroidTestRunner;
import android.test.InstrumentationTestRunner;
/**
* A test runner that each time runs the set of test executes them in random order.
* This might be useful to eliminate the dependencies between the test(s)
*
* @author marek.defecinski
*/
public class RandomizingTestRunner extends InstrumentationTestRunner {
private AndroidTestRunner androidTestRunner;
/**
* We hijack this method to make some modifications of the order of tests
*/
@Override
public void onStart() {
List<TestCase> testCases = androidTestRunner.getTestCases();
Collections.shuffle(testCases);
super.start();
}
/**
* When the {@link AndroidTestRunner android runner} is created in {@link InstrumentationTestRunner#onCreate(android.os.Bundle)} we save it here.
* The same method at the end has a start method which we override to randomize the order of the test
*/
@Override
protected AndroidTestRunner getAndroidTestRunner() {
androidTestRunner = super.getAndroidTestRunner();
return androidTestRunner;
}
}


Hope someone will find it useful.

sobota, 29 stycznia 2011

Working with signatures in apk(s)

Our task in the job was to write some (robotium) test to the application(s) that we were given only apk-s and the debug.keystore.

We had to use the same debug.keystore to be able to run InstrumentationTestCase. When the keys didn't match we get an error similar to this:

java.lang.SecurityException: Permission Denial: starting instrumentation ComponentInfo{net.retsat1.starlab.test/android.test.InstrumentationTestRunner} from pid=4745, uid=4745 not allowed because package net.retsat1.starlab.test does not have a signature matching the target net.retsat1.starlab


The keys have changed in the course of time few times and we soon get into the mess of not being sure which debug keystore was used.

We had to find a way to check which certificate is used.

On android pages you can get some information about signing the apks with the certificates.

http://developer.android.com/guide/publishing/app-signing.html#cert

and about security model used

http://developer.android.com/guide/topics/security/security.html


First thing we tried is the

jarsigner -verify -verbose -certs net.retsat1.starlab-1.apk


which output is

sm 520 Sat Jan 22 23:11:20 CET 2011 res/layout/main.xml

X.509, CN=Android Debug, O=Android, C=US
[certificate is valid from 05.10.10 21:47 to 05.10.11 21:47]


this can be compared to the output of the same command on the apk we are sure with which certificate was signed and look at the validity time. However, this is not very convenient. It would be better to get some number that can be easily compared.

Keytool is capable of displaying a certificate finger print

keytool -keystore ~/.android/debug.keystore -storepass android -list


That gives the output

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

androiddebugkey, 2010-10-05, PrivateKeyEntry,
Certificate fingerprint (MD5): 32:BD:9C:D1:1A:8C:32:95:B1:4F:14:45:B4:5D:86:D6

Which is basically what we were looking for. Now we needed the same for the apk.

After examining some robotium tutorials and this script sign-debug-any-apk.sh we produced a semi-automated script (printfingerprint.sh) for that.


#!/bin/bash

if [ $# -lt 1 ]; then
echo -e "Prints certificate fingerprint of the given apk file.\nUsage: $0 file.apk"
exit 1
fi

APKFILE=`readlink -f $1`

TEMP=`mktemp -d`
cd $TEMP
jar xf $APKFILE
keytool -printcert -file META-INF/CERT.RSA | egrep "^\s+MD5|^\s+SHA"
rm $TEMP -rf


Semi automatic as we need to get the apk from the device first.


printfingerprint.sh ~/tmp3/net.retsat1.starlab-1.apk

which leads us to

MD5: 32:BD:9C:D1:1A:8C:32:95:B1:4F:14:45:B4:5D:86:D6
SHA1: C4:68:47:DD:BF:61:C5:03:27:08:96:1C:7A:75:E1:42:DD:EC:C8:F2

and we can verify if the certificate in apk(s) match.