This week’s post is dedicated to reflection:
In computer science, reflection is the ability of a process to examine, introspect, and modify its own structure and behavior.
This is the 11th post in the Exercises in Programming Style focus series.Other posts include:
- Introducing Exercises in Programming Style
- Exercises in Programming Style, stacking things up
- Exercises in Programming Style, Kwisatz Haderach-style
- Exercises in Programming Style, recursion
- Exercises in Programming Style with higher-order functions
- Composing Exercises in Programming Style
- Exercises in Programming Style, back to Object-Oriented Programming
- Exercises in Programming Style: maps are objects too
- Exercises in Programming Style: Event-Driven Programming
- Exercises in Programming Style and the Event Bus
- Reflecting over Exercises in Programming Style (this post)
- Exercises in Aspect-Oriented Programming Style
- Exercises in Programming Style: FP & I/O
- Exercises in Relational Database Style
- Exercises in Programming Style: spreadsheets
- Exercises in Concurrent Programming Style
- Exercises in Programming Style: sharing data among threads
- Exercises in Programming Style with Hazelcast
- Exercises in MapReduce Style
- Conclusion of Exercises in Programming Style
Reflection in Python
The original Python solution uses reflection in different ways:
- The
exec()
function -
exec(object[, globals[, locals]])
This function supports dynamic execution of Python code. object must be either a string or a code object. If it is a string, the string is parsed as a suite of Python statements which is then executed (unless a syntax error occurs). If it is a code object, it is simply executed. In all cases, the code that’s executed is expected to be valid as file input. Be aware that the
return
andyield
statements may not be used outside of function definitions even within the context of code passed to theexec()
function. The return value isNone
.Simply put, the Python code is written in a string, and then passed to the
exec()
function to be evaluated e.g.:extract_words_func = "lambda name : [x.lower() for x in re.split('[^a-zA-Z]+', open(name).read()) if len(x) > 0 and x.lower() not in stops]" frequencies_func = "lambda wl : frequencies_imp(wl)" sort_func = "lambda word_freq: sorted(word_freq.items(), key=operator.itemgetter(1), reverse=True)" exec('extract_words = ' + extract_words_func) (1) exec('frequencies = ' + frequencies_func) (1) exec('sort = ' + sort_func) (1)
1 This defines a new function, which body is in the string above - The
locals()
function -
locals()
Update and return a dictionary representing the current local symbol table. Free variables are returned by
locals()
when it is called in function blocks, but not in class blocks.Interestingly enough, Python stores functions - and attributes - in a map. Remember that we did that explicitly in one earlier post.
word_freqs = locals()['sort'](locals()['frequencies'](locals()['extract_words'](filename)))
Our first taste of reflection
Using reflection can start quite easily - and we already did that in a previous post - by using function references:
fun stopWords() = read("stop_words.txt")
.flatMap { it.split(",") }
val funcStopWords = ::stopWords.invoke()
The usage of function references is not similar to the Python sample. As mentioned above, there are two aspects to reflection:
- Invocation through reflection
- Execution of valid Kotlin code encoded as string
Fortunately, Kotlin allows both. Let’s focus on the first part.
In the initial step, we need to move functions to a dedicated class.
class Reflective {
fun stopWords() = read("stop_words.txt").flatMap { it.split(",") }
// Other functions go here
}
In the next step, we shall use Kotlin’s Reflection API.
At the center of the API lies the KClass
, the equivalent of Java’s Class
.
Here’s a very simplified class diagram of the API:
The API allows this kind of code:
val clazz: KClass<out Any> = Class.forName("c.f.b.eps.Reflective").kotlin (1)
val funcStopWords: KFunction<*> = clazz.declaredMemberFunctions.single {
it.name == "stopWords" (2)
}
val constructor: KFunction<Any>? = clazz.primaryConstructor (3)
val instance: Any? = constructor?.call() (4)
val stopWords: List<String> = funcStopWords.call(instance) as List<String> (5)
1 | To get a KClass instance, Kotlin provides the kotlin extension property on the java.lang.Class class |
2 | Because functions are stored in a collection and not a map, one needs to filter out those whose name don’t match |
3 | The same applies to constructors. However, there’s a single primary constructor. |
4 | Creating a new instance is as simple as calling the constructor |
5 | Other functions can be called as well: the first object passed as a parameter is the instance the function will be called on |
Turning the tables
The above code is great, with one issue IMHO: we actually had to move the functions to a dedicated class. If this wouldn’t have been done, we would have got the following error at compile-time:
Packages and file facades are not yet supported in Kotlin reflection.
Interestingly enough, this is not the case with the Java Reflection API… in Kotlin. Hence, by migrating from Kotlin’s reflection to Java’s reflection, we could put the functions directly at the root! Let’s do that:
fun stopWords() = read("stop_words.txt").flatMap { it.split(",") }
Now, we need to use the exact counterpart of Kotlin’s API:
val clazz: Class<*> = Class.forName("ch.frankel.blog.eps._17_ReflectiveKt") (1)
val funcStopWords: Method = clazz.getMethod("stopWords") (2)
val words: List<String> = funcExtractWords.invoke(null, filename) as List<String> (3)
1 | Get a handle on the class.
Note that despite being at the root, functions will be generated as static methods of a class anyway.
There’s a default class name, but it can be overriden by using the @JvmName .
It’s not the case here though, therefore the convoluted name |
2 | The Class.getMethod() method is the traditional way to get a reference on a method
In Java, methods are held in a map, so it’s possible to get them directly, as opposed to Kotlin |
3 | Because root functions become static methods, the first parameter to invoke() needs to be null , as there’s no instance involved |
Source code generation and compilation
So far, we called code reflectively. However, we didn’t execute strings as Kotlin code as is the case in Python. In Python - and JavaScript - it’s easy, as those are interpreted languages. In Kotlin, however, source code must first be compiled before being executed.
Aye, there’s the rub: there’s no compilation API in Kotlin at the time of this writing, as opposite to Java (since 1.6). Therefore, it’s not possible - to the extent of my knowledge - to easily compile source code on the fly. An option is to use a build tool, with distinct modules:
- one to generate the source code
- one to compile it
- one to use it
String concatenation is not a really interesting way to generate source code. An alternative is to use a dedicated API for that, namely Kotlin Poet.
KotlinPoet
is a Kotlin and Java API for generating.kt
source files.Source file generation can be useful when doing things such as annotation processing or interacting with metadata files (e.g., database schemas, protocol formats). By generating code, you eliminate the need to write boilerplate while also keeping a single source of truth for the metadata.
I believe usage of Kotlin Poet fills the role of Python’s exec()
.
To use Poet, the architecture can be refined to the following two modules:
- Source code generation
-
This module uses the Kotlin Poet API to generate the source code through a Maven plugin.
- Main
-
This module calls the former plugin to generate the source code into an additional source folder. Generation needs to be done in an early phase of the lifecyle, before
compile
, e.g.generate-sources
. Then, compilation will manage the main source folder, as well as the additional one.
Kotlin Poet could be the theme of several blog posts. It’s centered around two principles, the Builder pattern and a fluent API.
Here’s a simplified class diagram:
Here’s an abridged sample, please check the Git repo for the whole source code:
val packageName = "ch.frankel.blog" (1)
val className = "Reflective" (1)
val stopWordsSpec = FunSpec.builder("stopWords")
.addCode("""return read("stop_words.txt").flatMap { it.split(",") }""") (2)
.build()
val file = FileSpec.builder(packageName, className)
.addType(
TypeSpec.classBuilder(ClassName(packageName, className))
.addFunction(stopWordsSpec)
.build()
)
.build()
file.writeTo(System.out) (3)
1 | Create constants to avoid repeating the string later |
2 | String blocks allow to use double quotes in the block code without escaping them |
3 | Dump the generated source code on the standard out. In "real" code, this gets written to a file. |
Executing the above code will print the following to the standard out:
package ch.frankel.blog
class Reflective {
fun stopWords() = read("stop_words.txt").flatMap { it.split(",") }}
Conclusion
Several ways are available to dynamically handle code at runtime. Some languages e.g. C, Scala and Clojure - offer macros. Macros allow some form of pre-processing of source code.
Other languages - e.g. Java and Kotlin (and Scala too) offer reflection. It provides the ability to access and execute code in a dynamic way at runtime. It also allows to generate source code. However, one should be aware that reflection comes at a definite performance cost.