Abstract
We identify a theoretical incompatibility between 3D point cloud masked autoencoding and SE(3)-equivariant networks. We prove that standard masking triggers a topological collapse, effectively forcing the model to hallucinate orientation without a reference frame. To resolve this, we introduce SIEVE (Scalar Invariant Extraction Via Equivariance), an SE(3)-equivariant self-supervised framework that compresses point cloud geometry into invariant scalar embeddings. At its core is the spherical projection mask, which preserves global pose while obfuscating local semantics, sidestepping the obstruction by construction. Experiments confirm that SIEVE circumvents the predicted collapse where constant and noise masks fail. On downstream point cloud tasks, the resulting embeddings improve over coordinate-based baselines and hand-crafted geometric descriptors, with the largest gains on generalization to unseen shape categories.
Why standard masking with equivariant networks fail
For point clouds, the coordinates are the sole feature encoding both where a point is and what the shape is. This motivates the use of SE(3)-equivariant networks that transform their geometric outputs consistently when the input rotates or translates.
Standard constant or randomized masking can erase the geometric information needed for reconstruction. We use vector-bundle analysis to characterize a topological obstruction for equivariant reconstruction: masks that discard this structure cannot reconstruct a broad class of non-trivial geometric representations.
How SIEVE works
- Encode the geometry. An SE(3)-equivariant encoder produces per-point scalar and vector output. SIEVE keeps the invariant scalar outputs as the embedding and discards the vector output.
- Preserve pose with a spherical projection mask. Each point is projected onto a unit sphere centered at the point cloud’s mean. This hides local shape details while preserving geometric orientation and position.
- Reconstruct point by point. The equivariant decoder combines the scalar embeddings with the masked coordinates to recover the original geometry. A pointwise L1 loss enforces correspondence with the input.
The implementation uses a Geometric Algebra Transformer (GATr) backbone. Gaussian perturbations and an additional decoder attention mask make reconstruction harder and help prevent a trivial radius-only encoding. The encoder’s scalar-input noise is separate from the noise added to projected coordinates.
Reconstructing different topologies
We compare spherical projection, constant vector, and ellipsoid noise masks on a synthetic dataset containing a solid cuboid and a square toroid. Each class has 100 training and 20 test examples. Both GATr and SE(3)-Transformer backbones are evaluated with matched initialization and hyperparameters.
As a control, the paper also studies DeepJEB objects with identical topology, where constant masking can reconstruct successfully. The failure is therefore tied to topological variety in these experiments, rather than a universal inability of the architecture to learn.
Pre-training and invariant embeddings
SIEVE is pre-trained on approximately 876k point clouds from OpenShape with 2,048 sampled points per cloud. The encoder learns a scalar embedding for every input point that is invariant to global rotation and translation.
Transfer to downstream tasks
The pre-trained encoder is frozen. Its pointwise scalar embeddings are concatenated with raw coordinates at the downstream model’s input, allowing existing architectures to use the representation without fine-tuning SIEVE.
Molecular segmentation
On RNA molecular segmentation, SIEVE improves accuracy across PointNet++, DeltaNet, and DiffusionNet compared with coordinate-only inputs and hand-crafted radius or curvature descriptors.
| Backbone | Coordinates | + Radius | + Curvature | + SIEVE |
|---|---|---|---|---|
| PointNet++ | 71.55 | 75.54 | 72.90 | 86.28 |
| DeltaNet | 87.21 | 88.07 | 86.86 | 89.58 |
| DiffusionNet | 81.20 | 85.89 | 85.97 | 89.30 |
3D shape completion
PCN is evaluated on eight seen and eight unseen 3D-EPN categories. Average Chamfer loss falls by 1.62% on seen categories and 12.5% on unseen categories. These are category averages; some individual categories have higher loss with SIEVE. This experiment uses the ShapeNet-pretrained encoder.
| Test categories | PCN | PCN + SIEVE |
|---|---|---|
| Seen | 4.6366 | 4.5615 |
| Unseen | 11.9848 | 10.4869 |
Geometric registration
On ModelNet40, DCP is evaluated with all 40 categories shared between training and testing, and with 20 training categories separated from 20 test categories. Adding SIEVE reduces rotation and translation error in both settings, with the larger rotation-error reduction on unseen categories.
| Category split | Model | Rotation RMSE | Translation RMSE |
|---|---|---|---|
| Shared | DCP | 1.214808 | 0.002040 |
| Shared | DCP + SIEVE | 0.838728 | 0.001539 |
| Separate | DCP | 2.635946 | 0.005048 |
| Separate | DCP + SIEVE | 0.935390 | 0.001509 |
The RNA and registration tasks use the OpenShape-pretrained encoder. These comparisons test transfer to established architectures; the full paper provides all category-level results, metrics, and ablations.
BibTeX
@inproceedings{lee2026sieve,
title={SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning},
author={Lee, Jongann and Park, Sun Woo and Choi, Yun Young},
booktitle={Advances in Neural Information Processing Systems},
year={2026},
eprint={2610.21087},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.21087}
}Acknowledgments
This research was supported by the Advanced GPU Utilization Support Program funded by the Government of the Republic of Korea (Ministry of Science and ICT). SolverX provided in-kind research support through GPU computing resources and LLM-based tools. Sun Woo Park thanks the Max Planck Institute for Mathematics for its support.