3 - Smoke: Bypassing StringFog and Decoding a Custom String Cipher

Overview

Smoke was a StringFog challenge. StringFog is an open-source Android string obfuscation tool that transforms every string literal in the bytecode into an encrypted hex payload, decoded on demand by a runtime method injected into the class. The goal is to make static analysis significantly harder: decompiling the APK produces no human-readable strings, only opaque calls like AbstractC0006a5.o("e30385842152b38cc3ab85").

The challenge used a custom variant of StringFog with its own cipher. The goal was to identify the encryption algorithm, port it to Python, and bulk-decrypt all string payloads in the decompiled source tree to find the flag.

What StringFog Does

StringFog works at the bytecode level. During the build or post-processing step, it:

1. Scans class bytecode for const-string instructions
2. Encrypts each string with the configured cipher
3. Replaces the const-string instruction with a call to a static decode method
4. Injects the decode method (and its supporting key material) into each class

The result is an APK where running JADX produces hundreds of call sites like:

1String s = AbstractC0006a5.o("68e42c1c401aad9a117be4c42ab2d...");

…and almost no readable strings anywhere in the output. The cipher in Smoke was custom — not the default XOR variant that ships with stock StringFog — which meant no off-the-shelf deobfuscator could handle it directly.

The Decoder Method

JADX named the class AbstractC0006a5 (a case-collision rename from the original a5). The static method o(String) was the sole entry point for all string decryption at runtime:

 1public static String o(String str) {
 2    if (str.isEmpty()) return "";
 3    byte[] bArr = new byte[8];
 4    long j = -2401050963399994828L;
 5    for (int i2 = 0; i2 < 8; i2++) {
 6        j = (j * 6364136223846793005L) + 1442695040888963407L;
 7        bArr[i2] = (byte)(j >>> 56);
 8    }
 9    int length = str.length() / 2;
10    int xorSum = 0;
11    for (int i4 = 0; i4 < length; i4++) {
12        xorSum ^= Integer.parseInt(str.substring(i4*2, i4*2+2), 16);
13    }
14    byte[] out = new byte[length];
15    for (int i = 0; i < length; i++) {
16        int b = Integer.parseInt(str.substring(i*2, i*2+2), 16);
17        int i8 = i + 1;
18        if (((i * i8) & 1) == 0) {
19            int i9 = (b - (bArr[i8 % 8] & 255)) & 255;
20            out[i] = (byte)((((i9 << 5) | (i9 >>> 3)) & 255) ^ (bArr[i % 8] & 255));
21        } else {
22            out[i] = (byte)((b ^ (bArr[i % 8] & 255)) ^ (xorSum & 255));
23        }
24    }
25    return new String(out, StandardCharsets.UTF_8);
26}

The algorithm had three distinct parts:

1. Keystream generation. Eight bytes are derived using a Linear Congruential Generator (LCG) with a hardcoded seed, multiplier, and addend — the same constants used in Java’s own Random class:

1seed = −2401050963399994828
2mul  =  6364136223846793005
3add  =  1442695040888963407
4keystream[i] = (LCG_next() >>> 56) & 0xFF

2. XOR checksum. A single checksum byte is computed by XOR-ing all input bytes together.

3. Position-dependent decode. For each byte at index i:

• If i * (i+1) is even: subtract a keystream byte, rotate left by 5 bits, XOR with another keystream byte.
• If i * (i+1) is odd: XOR with a keystream byte, then XOR with the checksum.

The keystream is constant across all invocations — the seed is fixed in the bytecode, so every string can be decrypted offline without running the app.

The Python Port

Because the keystream is constant, it can be pre-computed once and reused:

 1def _key_bytes():
 2    j = (-2401050963399994828) & ((1 << 64) - 1)
 3    mul = 6364136223846793005
 4    add = 1442695040888963407
 5    out = []
 6    for _ in range(8):
 7        j = (j * mul + add) & ((1 << 64) - 1)
 8        out.append((j >> 56) & 0xFF)
 9    return out
10
11def decode_abstractc0006a5(hex_str):
12    ks = _key_bytes()
13    data = bytes.fromhex(hex_str)
14    xor_sum = 0
15    for b in data:
16        xor_sum ^= b
17    out = bytearray(len(data))
18    for i, b in enumerate(data):
19        i1 = i + 1
20        if ((i * i1) & 1) == 0:
21            i9 = (b - ks[i1 % 8]) & 0xFF
22            out[i] = (((i9 << 5) | (i9 >> 3)) & 0xFF) ^ ks[i % 8]
23        else:
24            out[i] = (b ^ ks[i % 8]) ^ (xor_sum & 0xFF)
25    return out.decode('utf-8', errors='replace')

With the decoder in hand, the next step was to find every call site.

Scanning the Sources

solve_smoke_flag.py used a simple regex over all JADX-decompiled Java files:

1CALL_RE = re.compile(r'AbstractC0006a5\.o\("([0-9a-fA-F]+)"\)')

Running it across the Smoke_jadx/sources/ tree produced hundreds of decoded strings — URLs, error messages, UI labels, API keys — most of which were noise. The flag-shaped string (CTF{...}) appeared in exactly one place.

The Success Path

The flag was in ViewOnClickListenerC0106fa.java, in the button’s onClick handler:

 1public final void onClick(View view) {
 2    boolean valid = AbstractC0006a5.o("e30385842152b38cc3ab85")
 3                        .equals(this.a.getText().toString().trim());
 4    if (valid) {
 5        strO = AbstractC0006a5.o("589b03cc40523bebf18b6b54d09a43")
 6             + "\n" + AbstractC0006a5.o("98bbb364aaaa5d1c4985535cfa")
 7             + "\n" + AbstractC0006a5.o("68e42c1c401aad9a117be4c42ab2d49abb3c8524211abbabf11dab5e3852b3bc");
 8    } else {
 9        strO = AbstractC0006a5.o("c8136384e0bc352c416b6df6494a0512597b13ac98ec");
10    }
11    textView.setText(strO);
12}

Decoding the four ciphertexts in the success branch:

• "589b03cc40523bebf18b6b54d09a43" → "Access Granted!"
• "98bbb364aaaa5d1c4985535cfa" → "Your flag is:"
• "68e42c1c401aad9a117be4c42ab2d49abb3c8524211abbabf11dab5e3852b3bc" → the flag

The infer_real_flag function in the solve script identified this pattern automatically — it searched for any concatenated triple where the middle string equals "Your flag is:" and the third string starts with "CTF{".

The first ciphertext ("e30385842152b38cc3ab85") was the expected password — decoded, it was the string the user would need to type into the app to reach the success branch.

Why This Cipher Fails

The weakness here is that the keystream is fully deterministic and public. Any analyst with the APK can compute the eight keystream bytes offline. The cipher provides no runtime secret — no key derived from device state, server response, or user input. A single Python script and the JADX output is everything needed to decrypt the entire string table in under a second.

The challenge design was effective as a workshop exercise because it required careful reading of both the LCG seed values and the position-dependent decode path. Getting the rotation direction or the subtract-vs-XOR order wrong silently produces garbage output, so the implementation detail mattered.

Key Techniques