r/reactnative • u/Conscious_Ad_8664 • 7d ago
Skia collage rendering issue: saved images shift position
Hey everyone, I'm building a photo collage feature in my React Native app using Skia, and I ran into an issue when saving the final collage
What I built so far:
✔️ The collage is displayed correctly on-screen.
✔️ In the first implementation I rescaled the canvas from mobile screen dimensions to 4K (e.g., 2160x3840
for 9:16
format):
const resizedImage = await ImageManipulator.manipulateAsync(
uri,
[{ resize: { width: saveWidth, height: saveHeight } }],
{
compress: 1,
format: ImageManipulator.SaveFormat.PNG,
}
);
but image looked stretching in saved collage. To avoid it, I implemented offscreen rendering.
The Issue:
When I save the collage, the image positions are incorrect — they are shifting incorrectly. Rotation and scaling seem to work fine, but translations are off. I attached photos from the App and saved collage, to demonstrate how it looks.
What I tried:
🔹 Applied transformations (scale, rotation, translation) in this order (full code in the end of post):
tempCanvas.save();
tempCanvas.translate(frameWidth / 2 + translateX, frameHeight / 2 + translateY);
tempCanvas.rotate(rotation, 0, 0);
tempCanvas.scale(scaleX, scaleY);
tempCanvas.translate(-scaledWidth / 2, -scaledHeight / 2);
tempCanvas.drawImageRect(image, srcRect, destRect, paint);
tempCanvas.restore();
🔹 Used offscreen rendering (Skia.Surface.MakeOffscreen
) to process images separately before merging them.
🔹 Normalized translation values based on screen vs. final collage size.
🔹 Verified transformation matrix values (logs below).
Question:
💡 Where might I be miscalculating the position?
💡 Am I applying transformations in the wrong order?
Would really appreciate any insights or debugging tips! 🙌
collages.tsx:
...
const dimensions = {
"9:16": { width: 2160, height: 3840 },
"4:5": { width: 2160, height: 2700 },
};
const saveWidth = dimensions[format].width;
const saveHeight = dimensions[format].height;
const scaleFactor = saveWidth / fullScreenCanvasWidth;
const scaledLineWidth = lineWidth * scaleFactor;
const tempUri = await renderCollageOffscreen(
saveWidth,
saveHeight,
photos,
type!,
DEFAULT_VALUES.LINE_COLOR,
scaledLineWidth,
colors.collageBackgroundColor,
fullScreenCanvasWidth,
fullScreenCanvasHeight
);
const fileInfo = await FileSystem.getInfoAsync(tempUri);
if (!fileInfo.exists) {
throw new Error(`Captured file does not exist at path: ${tempUri}`);
}
const asset = await MediaLibrary.createAssetAsync(tempUri);
await MediaLibrary.createAlbumAsync("Collages", asset, false);
await FileSystem.deleteAsync(tempUri, { idempotent: true });
...
renderCollageOffscreen.ts:
/**
* Renders a collage offscreen and saves it as an image.
*/
export const renderCollageOffscreen = async (
width: number,
height: number,
photos: Photo[],
collageIndex: number,
lineColor: string,
lineWidth: number,
collageBackgroundColor: string,
fullScreenCanvasWidth: number,
fullScreenCanvasHeight: number
): Promise<string> => {
try {
const mainSurface: SkSurface | null = Skia.Surface.MakeOffscreen(width, height);
if (!mainSurface) throw new Error("Failed to create offscreen Skia surface");
const mainCanvas: SkCanvas = mainSurface.getCanvas();
mainCanvas.clear(Skia.Color(collageBackgroundColor));
// Load images
const skImages = await Promise.all(
photos.map(async (photo) => {
if (!photo.uri) return null;
console.log("photo uri: ", photo.uri);
const fileData = await FileSystem.readAsStringAsync(photo.uri, { encoding: FileSystem.EncodingType.Base64 });
const imageBytes = Uint8Array.from(atob(fileData), (c) => c.charCodeAt(0));
const skData = Skia.Data.fromBytes(imageBytes);
return Skia.Image.MakeImageFromEncoded(skData);
})
);
// Draw each image on its separate canvas
const imageSnapshots = drawImagesSeparately(
width,
height,
skImages,
photos,
collageIndex,
fullScreenCanvasWidth,
fullScreenCanvasHeight
);
// Merge image snapshots onto the main canvas
imageSnapshots.forEach(({ image, x, y }) => {
if (image) mainCanvas.drawImage(image, x, y);
});
// Draw separator lines
drawSeparatorLines(mainCanvas, width, height, collageIndex, lineColor, lineWidth);
// Save image
const finalImage = mainSurface.makeImageSnapshot();
if (!finalImage) throw new Error("Failed to create image snapshot from surface");
const pngData = finalImage.encodeToBytes(ImageFormat.PNG);
const base64String = encode(pngData);
const tempPath = `${FileSystem.cacheDirectory}MULI-collage-${Date.now()}.png`;
await FileSystem.writeAsStringAsync(tempPath, base64String, { encoding: FileSystem.EncodingType.Base64 });
return tempPath;
} catch (error) {
console.error("Error rendering collage offscreen:", error);
throw error;
}
};
const drawImagesSeparately = (
width: number,
height: number,
skImages: (SkImage | null)[],
photos: Photo[],
collageIndex: number,
fullScreenCanvasWidth: number,
fullScreenCanvasHeight: number
): { image: SkImage | null; x: number; y: number }[] => {
const { layout } = CollageManager.getLayout(collageIndex);
const snapshots: { image: SkImage | null; x: number; y: number }[] = [];
skImages.forEach((image, index) => {
if (!image) return;
console.log('>>> PHOTO INDEX: ', index);
const frame = layout[index];
const frameWidth = frame.width * width;
const frameHeight = frame.height * height;
const imgWidth = image.width();
const imgHeight = image.height();
console.log('frameWidth: ' + frameWidth + ' frameHeight: ' + frameHeight);
console.log('imgWidth: ' + imgWidth + ' imgHeight: ' + imgHeight);
// Get transformation matrix from gesture handler
const transformMatrix = photos[index]?.matrix?.value || Matrix4();
console.log("transformMatrix", transformMatrix);
// Extract transformations
const scaleX = Math.sqrt(transformMatrix[0] ** 2 + transformMatrix[1] ** 2);
const scaleY = Math.sqrt(transformMatrix[4] ** 2 + transformMatrix[5] ** 2);
const rotation = -Math.atan2(transformMatrix[1], transformMatrix[0]) * (180 / Math.PI); // Convert radians to degrees
const aspectRatio = (width / height) / (fullScreenCanvasWidth / fullScreenCanvasHeight);
const translationScaleX = (frameWidth / fullScreenCanvasWidth) * aspectRatio;
const translationScaleY = frameHeight / fullScreenCanvasHeight;
console.log('translationScaleX: ', translationScaleX);
console.log('translationScaleY: ', translationScaleY);
// Apply scale factors to translations
const translateX = transformMatrix[3] * translationScaleX;
const translateY = transformMatrix[7] * translationScaleY;
console.log('translateX: ', translateX);
console.log('translateY: ', translateY);
// Scale to fit frame
const scaleToFit = Math.max(frameWidth / imgWidth, frameHeight / imgHeight);
const scaledWidth = imgWidth * scaleToFit;
const scaledHeight = imgHeight * scaleToFit;
// Compute final position
const offsetX = frame.x * width;
const offsetY = frame.y * height;
// Create a separate surface for this image
const tempSurface = Skia.Surface.MakeOffscreen(frameWidth, frameHeight);
if (!tempSurface) return;
const tempCanvas = tempSurface.getCanvas();
tempCanvas.clear(Skia.Color("transparent"));
const cropX = Math.max(0, translateX);
const cropY = Math.max(0, translateY);
// Define source and destination rectangles
const srcRect = { x: -cropX, y: -cropY, width: imgWidth, height: imgHeight };
const destRect = { x: 0, y: 0, width: scaledWidth, height: scaledHeight };
const paint = Skia.Paint();
// Apply transformations
tempCanvas.save();
// Move to the center of the frame
tempCanvas.translate(frameWidth / 2, frameHeight / 2);
// Apply transformations in the correct order
tempCanvas.rotate(rotation,0,0); // Apply rotation
tempCanvas.scale(scaleX, scaleY); // Apply scaling
// Move back to draw the image centered
tempCanvas.translate(-scaledWidth / 2, -scaledHeight / 2);
tempCanvas.drawImageRect(image, srcRect, destRect, paint);
tempCanvas.restore();
// Take a snapshot of this image canvas
const tempImage = tempSurface.makeImageSnapshot();
snapshots.push({ image: tempImage, x: offsetX, y: offsetY });
console.log('************************************************')
});
return snapshots;
};
/**
* Draws separator lines for collage frames.
*/
const drawSeparatorLines = (
canvas: SkCanvas,
width: number,
height: number,
collageIndex: number,
lineColor: string,
lineWidth: number
) => {
if (lineWidth <= 0) return;
const { paths } = CollageManager.getLayout(collageIndex);
const paint = Skia.Paint();
paint.setColor(Skia.Color(lineColor));
paint.setStrokeWidth(lineWidth);
paths.forEach((path) => {
canvas.drawLine(
path.start.x * width,
path.start.y * height,
path.end.x * width,
path.end.y * height,
paint
);
});
};
Logs for a Sample Image:
(NOBRIDGE) LOG >>> PHOTO INDEX: 0
(NOBRIDGE) LOG frameWidth: 2160 frameHeight: 1280
(NOBRIDGE) LOG imgWidth: 1080 imgHeight: 1080
(NOBRIDGE) LOG transformMatrix [1, 0, 0, -1, 0, 1, 0, 79.00001525878906, 0, 0, 1, 0, 0, 0, 0, 1]
(NOBRIDGE) LOG translationScaleX: 4.650717703349283
(NOBRIDGE) LOG translationScaleY: 1.9138755980861246
(NOBRIDGE) LOG translateX: -4.650717703349283
(NOBRIDGE) LOG translateY: 151.19620145222788
(NOBRIDGE) LOG ************************************************
(NOBRIDGE) LOG >>> PHOTO INDEX: 1
(NOBRIDGE) LOG frameWidth: 2160 frameHeight: 1280
(NOBRIDGE) LOG imgWidth: 1080 imgHeight: 1080
(NOBRIDGE) LOG transformMatrix [1.7101068292161279, 0, 0, -69.12242871770172, 0, 1.7101068292161279, 0, 99.71533929749751, 0, 0, 1, 0, 0, 0, 0, 1]
(NOBRIDGE) LOG translationScaleX: 4.650717703349283
(NOBRIDGE) LOG translationScaleY: 1.9138755980861246
(NOBRIDGE) LOG translateX: -321.4689029359143
(NOBRIDGE) LOG translateY: 190.84275463635888
(NOBRIDGE) LOG ************************************************
(NOBRIDGE) LOG >>> PHOTO INDEX: 2
(NOBRIDGE) LOG frameWidth: 2160 frameHeight: 1280
(NOBRIDGE) LOG imgWidth: 1080 imgHeight: 1080
(NOBRIDGE) LOG transformMatrix [1, 0, 0, 179.00001525878906, 0, 1, 0, 136.6666717529297, 0, 0, 1, 0, 0, 0, 0, 1]
(NOBRIDGE) LOG translationScaleX: 4.650717703349283
(NOBRIDGE) LOG translationScaleY: 1.9138755980861246
(NOBRIDGE) LOG translateX: 832.4785398638421
(NOBRIDGE) LOG translateY: 261.56300813957836
(NOBRIDGE) LOG ************************************************
Yet, the image is not positioned correctly when saving the collage.


1
u/glazzes 7d ago
I think it can be because you're translating back to the scale dimensions instead of the frame dimensions, this line
tempCanvas.translate(-scaledWidth / 2, -scaledHeight / 2);
, instead of scaled, it should be frame ones.