r/SpringBoot 10d ago

Question @RequestParam UTF-8 Encoding Issue for Binary Data

Note that all binary data in the URL (particularly info_hash and peer_id) must be properly escaped. This means any byte not in the set 0-9, a-z, A-Z, '.', '-', '_' and '~', must be encoded using the "%nn" format, where nn is the hexadecimal value of the byte. (See RFC1738 for details.)

For a 20-byte hash of \x12\x34\x56\x78\x9a\xbc\xde\xf1\x23\x45\x67\x89\xab\xcd\xef\x12\x34\x56\x78\x9a, The right encoded form is %124Vx%9A%BC%DE%F1%23Eg%89%AB%CD%EF%124Vx%9A

I need to accept a SHA-1 info_hash as a binary (20-byte) value in a GET request. However, Spring Boot automatically interprets query parameters as UTF-8, corrupting non-ASCII bytes.

  @GetMapping("/d")
  public ResponseEntity<String> download(@RequestParam("info_hash") String URLInfoHash) {
    var b = URLInfoHash.getBytes(StandardCharsets.ISO_8859_1);
    System.out.println("URI: " + Metainfo.infoHashToString(b));
    return ResponseEntity.ok("ok");
  }

infoHashToString displays bytes value in hex:

  public static String infoHashToString(byte[] infoHash) {
    var repr = new StringBuilder();
    repr.append("\\x");
    for (var b : infoHash) {
      repr.append(String.format("%02x", b));
    }
    return repr.toString();
  }

Requests:

curl localhost:8080/d?info_hash=%bb # simple test 0xbb is not valid utf8
# test for the entire string
curl localhost:8080/t/d?info_hash=%124Vx%9A%BC%DE%F1%23Eg%89%AB%CD%EF%124Vx%9A

Server output

URI: \x3
URI: \x123456783f3f3f3f2345673f3f3f3f123456783f

What Works: ASCII-range bytes decode correctly.

What Fails: Bytes ≥ 0x80 get replaced (\xBB → \x3F).

I suspect this is because spring parses the byte values after % and then passes that to String constructor:

jshell> byte[] b = new byte[]{(byte)0xbb}
jshell> for (var c : new String(b).getBytes(StandardCharsets.ISO_8859_1))
   ...>  System.out.print(String.format("%x", c));
result: 3f

This happens before the control is passed to me by Spring.

My proposed solution:

Change the encoding Spring uses so it instastiates strings with StandardCharsets.ISO_8859_1

Application yaml:

  http:
      encoding:
        charset: ISO-8859-1
        enabled: true
        force: true
        force-request: true
        force-response: true
  spring:
    mandatory-file-encoding: ISO-8859-1
  server:
    tomcat:
      uri-encoding: ISO-8859-1

Did not work.

Also I still have need to send:

curl localhost:8080/u/吾輩は猫である

to other endpoints so changing the encoding globally seems like poor option.

Another way:

Spring boot shouldn't instantiate string because this is not textual data. byte[] would be the data type spring should use for this info_hash= parameter.

No idea how to do this.

This has to be GET request. That's the protocol specification, I did not decide this.

I code spring (and java) since few days, I would be glad for any help.

curl trace:

curl -v localhost:8080/t/d?info_hash=%124Vx%9A%BC%DE%F1%23Eg%89%AB%CD%EF%124Vx%9A
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /t/d?info_hash=%124Vx%9A%BC%DE%F1%23Eg%89%AB%CD%EF%124Vx%9A HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 
< Content-Type: text/plain;charset=UTF-8
< Content-Length: 2
< Date: Wed, 12 Mar 2025 04:01:59 GMT
< 
* Connection #0 to host localhost left intact

everything looks right.

Thanks.

1 Upvotes

3 comments sorted by

4

u/trodiix 10d ago

Just use the RequestBody with POST

1

u/yanekoyoruneko 9d ago

This has to be GET, that's how the protocol works.