README.md
Package detail
@needle-tools/gltf-build-pipeline
Pipeline and tools for optimizing gltf files using gltf-transform and compression settings within glTF extensions
readme
changelog
Changelog
All notable changes to this package will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
[2.9.5] - 2024-05-23
- change:
product
usecase usesUASTC
compression for 'normalTexture' - update gltf-transform to 4.1.3
[2.9.0] - 2024-05-22
- add: initial texture compression mode support for
world
andproduct
[2.8.1] - 2024-04-17
- bump version
[2.8.1-alpha.0] - 2024-03-26
- Fix: Issue where cache key gets longer depending on the primitive count
- Fix: Cache key not taking index buffer into account
- Fix: Caching of mesh LODs did clone shared mesh attributes for every primitive instead of re-using previous clones
[2.8.0-alpha.2] - 2024-03-04
- change: use webp compression for textures smaller than 32x32
- fix: webp compression for small textures (64x64)
[2.8.0-alpha.0] - 2024-02-27
- change:
product
usecase now uses webp compression for 'normalTexture' and 'metallicRoughnessTexture' textures
[2.7.1] - 2024-02-14
- fix: bug in caching path due to change in 2.7.0-alpha.2
[2.7.0] - 2024-02-13
- bump version
[2.7.0-alpha.2] - 2024-02-13
- fix: processing for some very large meshes where hashing the meshes failed (for caching the results)
[2.7.0-alpha.1] - 2024-01-13
- fix: handle LOD omitting and remove deleted files from progressive transform results list
[2.7.0-alpha] - 2024-01-13
- add: VRM mesh simplification support
- fix: issue where the output directory was not created if it didn't exist yet
- fix: try resolve deduplicated meshes
- change: update gltf-transform to 4.1.1
- change: modify mesh LOD settings for more vertices
[2.6.0-alpha] - 2024-12-16
- add: create placeholder textures for missing jpg, png, webp, gif, tif on disc
- fix: compressing a gltf file to a different output directory does not falsely delete old external textures on disc in the input directory
[2.5.0] - 2024-12-16
- release
[2.4.0] - 2024-09-11
- release
[2.4.0-beta.5] - 2024-09-06
- fix: issue where mesh LODs would in some cases be compressed with "draco" while the root file would be compressed with "mesh-opt" if not explicitly specified in the root file. This would cause invalid rendering and randomly scaled LOD meshes. Now mesh LODs will always use the same mesh compression. NE 5297
- change: update gltf-transform dependency to 4.0.8
[2.4.0-beta] - 2024-08-13
- fix: issue where a mesh lod is removed during second processing of a directory that contains existing mesh lods
[2.4.0-alpha.3] - 2024-08-11
- fix: issue where files would not correctly be overriden in subdirectories when no output directory was specified
[2.4.0-alpha.2] - 2024-06-29
- fix: prevent joining/flattening of progressive assets
[2.4.0-alpha.1] - 2024-06-26
- add: heuristic for choosing mesh compression if not explictly configured
[2.3.0-alpha.2] - 2024-06-26
- change: usecase
product
does compress with UASTC again (for certain slots)
[2.3.0-alpha.1] - 2024-06-23
- change: bump gltf-transform to 4.0.2 to fix meshopt skinning regression 1439
[2.3.0-alpha] - 2024-06-22
- add: flatten and join for
product
usecase
[2.2.1-alpha.8] - 2023-06-22
- add: CLI <output> support
- fix: register webp extension for valid glTF files. issue 5122
- fix: issue where VRM material would be pruned
- change: VRM use meshopt compression where any blendshapes are found
[2.2.1-alpha.7] - 2023-06-19
- fix: WARN if image validation can not run
- change: compress lightmap textures with webp
[2.2.1-alpha.5] - 2023-06-18
- add:
stats
now includes json and animation size info - change: VRM's are compressed with meshopt again
- change: usecase
product
now uses webp compression
[2.2.1-alpha.4] - 2023-06-17
- add:
--verbose
option to transform command - add:
stats
command to create stats.needle.json - add: automatically detect mimetype mismatch and attempt to fix it
[2.2.1-alpha] - 2023-06-15
- fix: issue with progressive mesh cache key calculation when running multiple transform processes with different configurations
[2.2.0-alpha.11] - 2023-06-14
- fix: issue where progressive transform with un-indexed meshes would fail
[2.2.0-alpha.10] - 2023-06-13
- fix: progressive transform of converted GLB files where meshes don't have indicies
- fix: skip progressive texture generation for textures for textures that already match the lowest LOD size
- fix: if compression is set to None the LOD textures should respect that setting and also not be compressed
- fix: fallback mesh density calculation for meshes that aren't indexed
- fix: NEEDLE_mesh_compression extension falsely overriding primitive extensions
[2.2.0-alpha.5] - 2023-06-06
- add: CLI
--progressive
option totransform
command (default is true) - add: CLI
--compress
option totransform
command (default is true) - add: config
meshes: { lods: boolean }
option - remove: CLI "compress" command
- change: enable texture LODs by default
- fix:
--config
path can now point to a directory or aneedle.config.json
file as well - fix: mesh LOD regression caused by change in gltf-transform
weld
behaviour - fix: KTX SOFTWARE >= 4.3.0 is installed
[2.1.0-alpha.6] - 2023-06-04
- fix: errors when invoking npm scripts caused by the cli program
- fix: remove cache directory error
- fix: remove dedup for textures since this currently causes issues with gltf-progressive loading
[2.1.0-alpha] - 2023-06-03
- add: compression config support for high-level usecases that can be passed via commandline arguments or via config. Available options are currently
default
,world
andproduct
. - add: texture LODs can now be generated by passing in the path to a compression config.
--config <path/to/config.json>
- change: VRMs now use draco mesh compression by default
Config Examples
default
- use default compression. When not explictly configured textures will be compressed with ETC1Sproduct
- when not explictly configured textures will use UASTC formetallicRoughnessTexture
andnormalTexture
world
- generate texture LODs when enabled (and not explictly disabled bytextures: { lods: false }
)
The config can also be part of a{ "usecase": "default" // optional "textures" : { "lods": true } }
needle.config.json
:{ ... regular needle.config "gltf": { "usecase": "default" // optional "textures": { "lods": true } } }
[2.0.0-alpha] - 2023-06-01
- update gltf-transform dependency to 4.x
change: ensure ETC1S and UASTC textures are power of 2
CLI: commands can be run with
needle-gltf <command>
- CLI: add
--cache
to allow disabling cache with--cache False
(default true) - CLI: add
--stats
to produce.needle.stats.json
(default false) - CLI: change
--output
to be the optional second argument
[1.6.0-alpha.7] - 2023-05-30
- fix: npx cli command return wrong error due to
packGLTF
not returning a booelan - fix: mjs script import paths
[1.6.0-alpha.5] - 2023-05-28
- fix: determine build pipeline version
- change: throw error and exit process if packing or transform fails
[1.6.0-alpha.2] - 2023-05-27
- add:
gltf-build-pipeline
commandline support withtransform
andcompress
(pack-gltf) options.
[1.5.0-alpha.12] - 2023-05-20
- add: pack-gltf now also runs
resample
for animations - fix: nullreference error in accessing opaque debug options
[1.5.0-alpha.10] - 2023-05-17
- add:
generateLods
option to progressive texture settings
[1.5.0-alpha.9] - 2023-05-16
- add: start of VRM 1.0 support. Not implemented is the
VRMC_materials_mtoon
extension
[1.5.0-alpha.8] - 2023-05-16
- fix: issue where updating ktx compressed texture extension also flattened the path
[1.5.0-alpha.7] - 2023-05-16
- fix: external resources that are KTX2 compressed are now saved with a
.ktx2
extension - change: toktx now uses compression "auto" mode to select uastc or etc1s
- change: toktx now checks the slots and increases quality for
metallicRoughnessTexture
andnormalTexture
but keeps using ETC1S for smaller file size
[1.5.0-alpha.6] - 2023-05-15
- fix: If not further specified then automatically use UASTC for textures that are in either
normalTexture
ormetallicRoughnessTexture
slots
[1.5.0-alpha.5] - 2023-05-04
- fix: NEEDLE_opaque should handle if VRM texture index is negative
- change: exit with code 1 if any file packing fails
[1.5.0-alpha.4] - 2023-05-03
- fix: issue in NEEDLE_opaque resolving of VRM textures, also resolve
meta.texture
- fix: VRMs now compress meshes with mesh-opt if nothing is specified
- change: include version in cache directory
[1.5.0-alpha.3] - 2023-05-03
- add: VRM progressive textures support
- fix: VRM format should be saved as GLB. See https://github.com/donmccurdy/glTF-Transform/issues/1390
- fix: VRM texture index handling
[1.5.0-alpha] - 2023-05-01
- add: NEEDLE_opaque
node: <index>
retargeting - add: passthrough VRM extension
[1.4.11] - 2023-04-29
- add: NEEDLE_progressive mesh now run sparse transform
- change: cli params for debug and caching are now
--debug
and--no-caching
[1.4.10] - 2023-04-23
- add: NEEDLE_progressive can now load and process already draco compressed meshes
[1.4.9] - 2023-04-23
- add: NEEDLE_progressive now have
hash
field for LODs which is generated using the content of the LOD and it's settings that produced it. - fix: NEEDLE_progressive catches error when copying mesh with extensions where the copy operation fails
[1.4.8] - 2023-04-22
- fix: update sharp package to
^0.32.6
- change: NEEDLE_progressive meshes max LOD error increased
[1.4.7] - 2023-04-16
- add:
asset.extras.needle
now contains information about used build pipeline
[1.4.6] - 2023-04-10
- fix: NEEDLE_progressive meshes do not revert primitive data for skinned meshes anymore. See NE-4596
[1.4.5] - 2023-04-05
- fix: generating mesh and texture LOD for glbs without
NEEDLE_
extension - fix: guid generation for progressive meshes when no
NEEDLE_
extension is present
[1.4.4] - 2023-04-03
- change: clamp simplification error for LOD meshes to 5%
[1.4.3] - 2023-04-02
- release
- fix: generate cache key for meshes
[1.4.1-beta.15] - 2023-04-02
- fix: issue where discarding all LODs might result in broken mesh due to simplify having modified the original mesh
- fix: don't produce LODs if GLB/glTF was produced with old generator
[1.4.1-beta.13] - 2023-03-26
- change: passthrough
NEEDLE_editor
&NEEDLE_gltf_dependencies
extensions
[1.4.1-beta.12] - 2023-03-23
- add: NEEDLE_progressive LODs can now contain more info per texture or mesh. Meshes contain vertex- and index counts
- change: the cache can now store and retrieve meta information
[1.4.1-beta.11] - 2023-03-20
- add: NEEDLE_progressive LOD meshes morph targets data
- fix: NEEDLE_progressive should check if simplifcation did result in improved meshes (> 32 verts removed && > 5 % improvement) and otherwise discard LODs (and the whole extension if no improvement was made)
- change: Unity NEEDLE_progressive logs
[1.4.1-beta.10] - 2023-03-19
- change: NEEDLE_progressive calculates density per mesh level
- change: NEEDLE_progressive
uri
field is removed and addedlod
array of objects
[1.4.1-beta.8] - 2023-03-18
- change: if an asset doesnt have a guid we use a uuid package now
- change: ignore mesh LODs with vertex count < 4
- change: allow generating mesh LODs without extension (if the mesh has > 32 vertices)
- change: omit generated LODs if the change is too small (e.g. the vertex count didnt change enough)
[1.4.1-beta.6] - 2023-03-15
- add: caching support for mesh LODs
[1.4.1-beta.5] - 2023-03-15
- add: auto LODs now calculate mesh density
- change: generate 6 LOD level
- change: use ratio 0.5 of the previous LOD level
- change: bump gltf-transform to 3.10.1 to fix simplifier
- change: add sparse transform to progressive transform
[1.4.1-beta.3] - 2023-03-14
- fix: wait for mesh simplifier to be ready
[1.4.1-beta.2] - 2023-03-13
- change: update gltf-transform to
^3.10.0
- change: progressive mesh transform does now set ratio for LODs (see simplifier issue 661 on github)
- fix: issue where progressive meshes were not generated if a mesh was multiple times in the glTF
- change: simplify to use ratio, fix issue where not all meshes were processed
- change: auto LOD max error set to 0.1
[1.4.0-beta] - 2023-03-12
- add: progressive mesh transform to automatically generate mesh LOD levels (currently we only generate one low poly version)
[1.3.1] - 2023-10-20
- fix: meshopt with khr_animation_2 extension was not correctly processed: animated node and target were not passed to gltf-transform which caused a wrong processing of meshopt. We now pass this information to the gltf-transform AnimationChannel object
- change: bump gltf-transform to 3.7.3
[1.2.4] - 2023-09-30
- fix: bug in progressive texture transform where extension wasnt added correctly (NE-3627)
[1.2.3] - 2023-09-29
- fix: bug in progressive texture transform where texture wasnt found
[1.2.2] - 2023-08-14
- remove: legacy squoosh dependency
[1.2.1] - 2023-07-25
- change: set reference to materials used by json pointers
[1.2.0] - 2023-07-21
- add: resolving of material json pointers
[1.1.6] - 2023-07-10
- change: bump gltf-transform dependency to latest 3.4.6
[1.1.5] - 2023-06-26
- change: bump gltf-transform dependency to latest 3.4.3
[1.1.4] - 2023-06-09
- fix: progressive texture transform didnt pass mesh compression setting through
[1.1.3] - 2023-05-24
- minor change adding info about reduced size after processing gltf files
[1.1.2] - 2023-05-24
- fix: wasm out of memory error caused by starting to many compression processes at once
[1.1.1] - 2023-05-18
- fix: fix progressive texture processing for empty slots array, assume texture slot "baseColorTexture" (for custom shader support)
[1.1.0] - 2023-05-06
- add: support for per glb mesh compression selection (none, draco or meshopt)
[1.0.6] - 2023-05-06
- fix: dont add textures to cache when toktx is not installed
[1.0.5] - 2023-05-06
- fix: catch exception in cases where toktx is not installed (e.g. in CI environment) and log warning instead
[1.0.4] - 2023-04-24
- fix: error caused by missing simplifier
[1.0.3] - 2023-04-24
- fix: webp quality setting not being rounded resulting in compression to fail
[1.0.2] - 2023-04-19
- change: progressive should not log error out when running for already compressed gltf
- change: ignore EXT_texture_exr extension re-assignment
[1.0.1] - 2023-04-17
- fix: register meshopt decoder
- fix: catch exception in document read
- change: log gpu memory estimation for textures
[1.0.0] - 2023-04-04
- change: file utils use needle.config.json to find files if no directory was specified
- change: add generator info
- change: try catch toktx exception
- change: improve relative path handling
- change: use env variable set by npm to resolve relative paths
- add: option to disallow toktx using
no-toktx
commandline parameter - update to gltf-transform 3.1.4
[0.11.2-pre] - 2023-01-30
- fix: clean up logic for finding toktx to only use one approach
[0.11.1-pre] - 2023-01-27
- fix: getting correct gltf transform texture for texture definition
- fix: opaque extension now also uses getTexture util method
[0.11.0-pre] - 2023-01-25
- add: script
transform
does run progressive and gltf-pack - change: script
transform:progressive
can now process all files in passed in directory
[0.10.10-pre] - 2023-01-25
- add:
NEEDLE_TOKTX
environment variable - fix: env variable handling
[0.10.9-pre] - 2023-01-20
- fix: set mime-type to KTX2 when using cached texture data that was processed with default toktx settings
[0.10.8-pre] - 2023-01-18
- fix: pack-gltf should run toktx with default settings when no compression extension was found in texture
[0.10.7-pre] - 2023-01-18
- fix: pack-gltf when invoked with single glb path
[0.10.6-pre.1] - 2023-01-13
- fix: support for glbs using old texture pointer format
textures/
(new format is/textures/
)
[0.10.5-pre] - 2023-01-12
- fix: pack-gltf running in sequence again to avoid wasm memory error with many glbs
[0.10.4-pre] - 2023-01-11
- change: simplify mesh logs
lockBorder
setting - change: pack-gltf now searches glb files in subdirectories
[0.10.3-pre] - 2023-01-09
- fix: issue with pack-gltf running out of memory when invoked for directory with many glbs
- change: await squoosh pool close
- change: disable extension read-write logs by default
[0.10.2-pre] - 2023-01-07
- add: mesh simplification now welds vertices with tiny threshold to be able to simplify disjointed meshes
- change: progressive loading handling of unknown slot, dont throw for unknown texture slot but instead log warning and proceed without making the texture progressively loaded
- change: update gltf-transform dependency to 2.5.1
[0.10.1-pre] - 2023-01-04
- fix: error in progressive texture resize where
maxSize
could be 0
[0.10.0-pre] - 2023-01-04
- add: caching for texture transforms
- change: use one needle_transform function to handle compressions of textures in one loop instead of mulitple toktx and squoosh transform methods
[0.9.1-pre] - 2023-01-03
- add: opaque extension support for primitive extensions to support mesh compression settings passed through when using progressive transformations
[0.9.0-pre] - 2023-01-03
- add:
NEEDLE_mesh_compression
support for running gltf-transform simplify per primitive
[0.8.1-pre.1] - 2022-12-14
- add: quality option to
NEEDLE_compression_texture
for webp compression - fix: progressive glb should always use name from guid and index
[0.8.0-pre] - 2022-12-13
- add: webp compression mode
- change: progressive texture glbs use guid as filename
- change: pack-gltf can now take directory as argument
[0.7.1-pre] - 2022-12-08
- fix: processing gltf multiple times did cause texture json pointers being resolved wrongly including loosing previously set extensions
[0.7.0-pre] - 2022-12-05
- add: make-progressive tool to transform an existing glbs and resize images with high res images being loaded from external glbs
[0.6.4-pre] - 2022-12-03
- add: compression mode
none
to skip toktx compression - fix: issue with texture pointer not finding previously assigned sampler when writing
- fix: if commandline parsing fails take path containing
KTX-Software
as additional toktx search path
[0.6.3-pre] - 2022-10-27
- fix: options.file nullreference error
[0.6.2-pre] - 2022-10-20
- change: toktx temp folder adjusted, use uuid for shorter names, generate uuid once per file PR
[0.6.1-pre] - 2022-10-20
- fix: NEEDLE_compression_settings havent been passed to toktx anymore
[0.6.0-pre] - 2022-10-19
- add: generate toktx textures in
glTF-transform
subdirectory, input and output textures having the same random name and are prefixed with the texture index
[0.5.1-pre] - 2022-10-12
- fix texture extensions surviving multiple compressions (e.g. compression same glb multiple times)
- fix cli args when invoked via npm run
[0.5.0-pre] - 2022-10-11
- add draco support again
[0.4.1-pre] - 2022-10-11
- fix texture extension pass-through
[0.4.0-pre] - 2022-10-11
- update toktx to gltf-Transform v2.4.4
[0.3.0-pre] - 2022-10-06
- add: handle NEEDLE_compression_texture extension and API to toktx to control cli arguments per texture
[0.2.2-pre] - 2022-10-03
- add: allow setting
--toktxPath
from commandline on windows
[0.2.1-pre] - 2022-10-03
- fix bug where EXT_texture_exr extension would be inserted in all textures
[0.2.0-pre] - 2022-10-02
- handle KHR_animation_pointer
- handle texture json pointers
[0.1.3-pre] - 2022-09-26
- initial release