Estimating the Android Architecture migration for Simple

12 minute read
Mobius 3 column

When we first started building the Simple Android app (Source), we chose an architecture that would let us move fast and implement new screens quickly, while we figured out what the product should be. Over time, as we moved from building new screens to making changes to the existing ones, the architecture we chose got in the way and resisted change.

We therefore needed a new architecture—one that would be more suited for long term maintenance. After investigating multiple approaches, we ultimately picked one based on Mobius, a reactive state management framework by the folks at Spotify.

Once we decided on the architecture itself, we came up with a migration guide which a developer could follow to migrate a screen in our original architecture to the new one. However, the app had grown to a considerable size and had many screens. We needed a way to estimate how long the migration process would take for better product planning.

Why estimate?

When it comes to architecture migrations, having a sense of how much effort is involved is critical for project planning.

Knowing that a new feature is going to be prioritised soon could help us make the decision on which screens to migrate first. Since we would want to implement new features in the current architecture instead of the old one, this made more sense. This also lets us provide better release dates for upcoming features because we have some idea of how much time it would take to migrate a screen.

It can be tempting to provide a vague estimate based on the number of screens. However, estimations based on feeling are rarely reliable, even if artificially inflated with some sort of buffer time. When estimations are repeatedly off by large amounts and impacts product roadmaps, it eventually leads to loss of trust in the engineering team, which is an undesirable outcome.

Therefore, we needed a reasonably reliable method to estimate the overall effort required to migrate all our screens and this method needed to be driven by data in some form. This article is the story of how we managed to get fairly accurate estimates for the migration process.

Finding a good heuristic for estimating

Initially, we migrated a few screens manually to get a sense of how long this would take. And after doing so, we noticed something that was quite... obvious. A screen that did more stuff, i.e, was more complex, took longer to migrate versus simpler screens. What we needed to do, then, was to find a way to compute a "complexity" score for each of our screens. We could then look at the time it took for us to migrate screens of different complexities, and accordingly come up with an estimate for how long other screens with similar complexities would take.

Let's take a look at a typical Controller type in our original architecture. This class was responsible for:

  • Taking in UI input events
  • Deciding what to do based on the UI events received
  • Doing the actual work itself by performing any required database queries and transformations
  • Finally, updating the UI with the results

This class was implemented as an ObservableTransformer in RxJava.

An example implementation for the screen that handles our "login" flow looked something like this:

typealias Ui = LoginPinScreen
typealias UiChange = (Ui) -> Unit

class LoginPinScreenController @Inject constructor(
    private val userSession: UserSession,
    private val requestLoginOtp: RequestLoginOtp,
    private val facilitySync: FacilitySync
) : ObservableTransformer<UiEvent, UiChange> {

  override fun apply(events: Observable<UiEvent>): ObservableSource<UiChange> {
   return Observable.mergeArray(
        screenSetups(events),
        backClicks(events),
        readPinDigestToVerify(events),
        syncFacilitiesAndLoginUser(events)
    )
  }

  private fun screenSetups(events: Observable<UiEvent>): Observable<UiChange> {
    // Method implementation
  }

  private fun backClicks(events: Observable<UiEvent>): Observable<UiChange> {
    // Method Implementation
  }

  private fun readPinDigestToVerify(events: Observable<UiEvent>): Observable<UiChange> {
    // Method implementation
  }

  private fun syncFacilitiesAndLoginUser(events: Observable<UiEvent>): Observable<UiChange> {
    // Method implementation
  }

  // Other methods
}

Looking at the structure of a controller, two things stand out as a good measure of how complex a controller is:

  1. The number of dependencies it has - the more the number of dependencies, the more work this controller is doing.
  2. The number of individual RxJava streams that are eventually merged as part of the apply() method. An individual stream represents one user flow that the screen handles, so the number of discrete user flows that the controller handles contributes to the overall complexity.

Eventually, the heuristic we decided to go with was to multiply the number of dependencies with the number of RxJava streams being merged:

Complexity = Number Of Dependencies * Number Of RxJava Streams

However, doing this manually would have been unfeasible since we had many screens in the app. But here's how we managed to get this data for the entire project automatically:

Code is data

Before we talk about how we accomplished this task, I'd like to take a moment to talk about something which might seem unrelated: JSON, what exactly is it?

JSON is a format for data exchange, where we can specify information in a standard text file in a well-understood structure. Specialised programs called parsers can then read and interpret this textual data and convert it into information that other programs can consume.

Similarly, the code that we write is also a format (albeit far more complex than JSON) for specifying information. The Kotlin compiler is able to read and interpret this textual data and use it to construct an application that gets run on devices.

There are tools available written in Kotlin for parsing Kotlin source code. One such tool is kotlinx.ast, which allows us to write a Kotlin program that:

  • Can read Kotlin source code and generate an Abstract Syntax Tree, or an AST, representing that code.
  • Process the generated AST to read what information is relevant to the task we are trying to accomplish.

The task we were trying to accomplish was finding out the complexity of every screen controller that was present in our code base. In order to achieve this, we broke the problem down into the following discrete steps:

  1. Find all screen controllers in the code base.
  2. For each controller, find the constructor and from it, the number of dependencies it has.
  3. For each controller, find the number of RxJava streams that are getting merged into the main .apply() method.
  4. Generate a complexity score for each controller by multiplying the numbers found in steps 2 and 3.
  5. Output the results in some format. We chose a CSV (comma separated value) file for easy import into spreadsheet software.

While steps 4 and 5 are fairly trivial to accomplish, we can talk about how the first three steps looked like to implement using the tool.

Before proceeding with the next section, we recommend going through the kotlinx.ast documentation. The following section will assume the reader has some idea of how kotlinx.ast works and is familiar with its usage.

Finding all screen controllers

This was easy to accomplish because we have a disciplined team that follows the patterns that we set in place. Every screen controller that was present has the following characteristics:

  • The name always has the "Controller" suffix.
  • It implements ObservableTransformer<UiEvent, UiChange>.

Using these two pieces of information, we were able to traverse the entire source folder to find the screen controllers:

val controllers = sourceFolder
        .walkTopDown()
        .filter { it.isFile && it.name.endsWith("Controller.kt") }
        .map { it.nameWithoutExtension to AstSource.String(it.readText()) }
        .map { (name, source) -> name to KotlinGrammarAntlrKotlinParser.parseKotlinFile(source) }
        .map { (name, source) -> name to source.summary().get() }
        .filter { (name, astList) -> astList.klassDeclaration(name).isActualController() }

This is what the isActualController() method looks like:

private fun KlassDeclaration.isActualController(): Boolean {
    return inheritance.find { it.children.any(Ast::isRxTransformer) } != null
}

private fun Ast.isRxTransformer(): Boolean {
    return this is KlassIdentifier && rawName.equals(
        "ObservableTransformer<UiEvent, UiChange>",
        ignoreCase = true
    )
}

Now that we have the list of screen controllers, we can proceed to the next part of the puzzle.

Finding the constructor and dependencies

In kotlinx.ast,, anything related to declaration of a type is represented by the kotlinx.ast.common.klass.KlassDeclaration type, which has a list of kotlinx.ast.common.ast.Ast children. The kotlinx.ast.common.ast.Ast type is itself an interface which has a deep class hierarchy, representing all the different components that go into a class declaration.

What we were interested in was finding out exactly which child of that entire list of children in the KlassDeclaration represented the constructor declaration. This was made easy by the fact that the kotlinx.ast.common.ast.Ast type imposed the requirement of a description property on every implementation. For a constructor, this will always be a constant "KlassDeclaration(constructor)". In addition, we had the added benefit of only having a single constructor for all our controllers, so this was fairly trivial once we found this out.

private fun KlassDeclaration.konstructorDeclaration(): KlassDeclaration {
    return children.find { it.description == "KlassDeclaration(constructor)" } as KlassDeclaration
}

When it comes to finding the number of dependencies accepted via the constructor kotlinx.ast represents these dependencies as KlassDeclaration types and sets a property called keyword to val. These declarations are children of the constructor AST we found earlier. We figured this out by using standard IDE debugging tools to examine the generated class hierarchy and examining it.

For example, taking the same controller constructor declaration as earlier,

class LoginPinScreenController @Inject constructor(
    private val userSession: UserSession,
    private val requestLoginOtp: RequestLoginOtp,
    private val facilitySync: FacilitySync
) : ObservableTransformer<UiEvent, UiChange>
Screenshot&#x20;from&#x20;2021&#x20;05&#x20;13&#x20;09&#x20;35&#x20;10

Once we found this out, finding the number of controller dependencies was fairly trivial.

private fun KlassDeclaration.findNumberOfDependencies(): Int {
    return children
        .filter { it is KlassDeclaration && it.keyword == "val" }
        .count()
}

Finding the number of RxJava streams

This method required us to dig into the class body. In kotlinx.ast, the class body is represented as an AstNode with a property description set to classBody. The class methods are represented as KlassDeclaration types with the keyword set to fun and are registered as children of the classBody node we found earlier. The next step was finding out which methods were the ones we were interested in.

Screenshot&#x20;from&#x20;2021&#x20;05&#x20;13&#x20;10&#x20;01&#x20;18

This was easier for us because the team was quite disciplined in maintaining certain coding conventions. This can be seen in the method declaration described earlier:

private fun screenSetups(events: Observable<UiEvent>): Observable<UiChange> {
    // Method implementation
}

private fun backClicks(events: Observable<UiEvent>): Observable<UiChange> {
  // Method Implementation
}

private fun readPinDigestToVerify(events: Observable<UiEvent>): Observable<UiChange> {
  // Method implementation
}

private fun syncFacilitiesAndLoginUser(events: Observable<UiEvent>): Observable<UiChange> {
  // Method implementation
}

As can be seen, all of the RxJava streams that contribute to the controller return an Observable<UiChange>. So the only thing we had to do then was to count the methods which returned Observable<UiChange> and we were good to go.

This is what we ended up using to find the number of RxJava streams in the end.

private fun KlassDeclaration.findNumberOfRxStreams(): Int {
    val node = this.expressions.find { it.description == "classBody" } as DefaultAstNode?

    return node
        ?.children
        ?.filterIsInstance(KlassDeclaration::class.java)
        ?.count { it.type?.rawName == "Observable<UiChange>" && it.identifier?.rawName != "apply" }
        ?: 0
}

We did add a special exclusion for the primary .apply method that is required by the ObservableTransformer contract (which our controllers have to implement), but even if we hadn't, it would not have made much of a difference since this difference would be applied in every controller, and what we needed was to figure out relative complexity.

Generating a complexity score

After extracting the important information, the only thing left to do was to compute a complexity score for each controller. This was quite trivial as we only had to multiply the number of dependencies with the number of RxJava streams, as mentioned earlier. For ease of processing, we chose to represent this information as a Record type and encapsulated the relevant logic within it.

data class Record(
    val name: String,
    val numberOfDependencies: Int,
    val numberOfRxStreams: Int
) {
    companion object {
        fun from(name: String, astList: List<Ast>): Record {
            val klass = astList.klassDeclaration(name)
            val konstructor = klass.konstructorDeclaration()

            return Record(
                name = name,
                numberOfDependencies = konstructor.findNumberOfDependencies(),
                numberOfRxStreams = klass.findNumberOfRxStreams()
            )
        }
    }

    val complexity: Int
        get() = numberOfRxStreams.coerceAtLeast(1) * numberOfDependencies.coerceAtLeast(1)

    fun toCsvRow(): String = "$name,$numberOfDependencies,$numberOfRxStreams,$complexity"
}

Exporting a CSV

The last remaining piece was to put everything we had together to export a CSV file, which we could then import into a spreadsheet tool. This is what the final piece of code looked like when everything was done:

fun main(args: Array<String>) {
    val sourceFolder = File(args[0], "app/src/main")

    val records = sourceFolder
        .walkTopDown()
        .filter { it.isFile && it.name.endsWith("Controller.kt") }
        .map { it.nameWithoutExtension to AstSource.String(it.readText()) }
        .map { (name, source) -> name to KotlinGrammarAntlrKotlinParser.parseKotlinFile(source) }
        .map { (name, source) -> name to source.summary().get() }
        .filter { (name, astList) -> astList.klassDeclaration(name).isActualController() }
        .map { (name, astList) -> Record.from(name, astList) }
        .onEach { println("Processed ${it.name}...") }
        .toList()

    val classesComplexityCsv = records
        .sortedByDescending(Record::complexity)
        .joinToString(
            separator = "\\n",
            prefix = "Name,Dependencies,Rx Streams,Overall complexity\\n",
            transform = Record::toCsvRow
        )

    val file = File("results.csv").apply { writeText(classesComplexityCsv) }

    println("Wrote results to ${file.absolutePath}")
}

This ended up giving us a great way to visualise the data in Google sheets, and use their tools to easily see which screens are going to require the most effort to migrate.

Mobius&#x20;asset&#x20;1

Results

Once we had this sheet in place, we migrated a few screens of varying complexities and tracked the time it took for us to do this, and then created complexity buckets based on that time. This looked something like:

Screenshot&#x20;2021&#x20;05&#x20;27&#x20;at&#x20;11&#x20;11&#x20;18&#x20;AM

While these were not the exact numbers we used, it serves as a good indication of what we did.

Given that we now had a complexity score for each screen, we were able to come up with a rough estimate for how long it would take to migrate all our screens to the new architecture. As per our estimation, it would take us around six months for a single engineer dedicated full time to complete this migration. We were reasonably accurate, because it took us around seven months to complete the migration in actuality.

It is to be noted here that all this was done in parallel with adding new features to the app. Having some heuristic for estimation was extremely helpful during project planning.

Takeaways

  • Meta-programming is a great (and often overlooked) tool in a programmer's toolbox, and can prove to be really helpful in a lot of situations.
  • Following code conventions and consistency may not seem important. But if the team hadn't been disciplined enough to do this, trying to use this technique to extract useful information from the code would have been a difficult order of magnitude since we have to handle different ways of representing controllers and user flows.

For those who might be interested, you can find the source code for the tool here: https://github.com/simpledotorg/simple-android-dependency-check/