I’m not an expert, but I have some related experience. could be that what I know is only true to a specific case, so if someone has a different answer, if doesn’t necessarily conflict. With that out of the way…
Pictures aren’t magically turn into a specific format, something has to do the work - getting the data from the sensor, compress it, add meta data, maybe do some other processing and pass it on to be saved.
Somebody has to make it happen. you can either pay some person in company to learn the documentation and implement it (write the code). this cost the time of development and maybe some license or support fee. The other way is to buy IP (intellectual property). basically this means buying the code to do the work then imbed it in the project. this, of course, cost money.
A company developing a new phone needs to balance the cost of implementing a solution - what is most convenient for the end users, what do they need, what do they want, what is the demand for new features and how much does is cost to implement. maybe they already have the code for something similar and only need to do small modifications and integration.
Now I bet JPEG is the cheapest, probably tons of IPs floating around, older implementations are a certainly for big companies. users know it and it just works for them 99.9% of the time.
Basically, why do something new if the old works and potentially cost more money to do.
Open source libraries doesn’t mean it’s free.
Firstly, even open source can have commercial licensing and can be free only for non-commercial use.
Secondly, the process of producing an image from the sensor isn’t a straight forward process, the compression and containerization is only one part. In order to get the final picture all sort of processing is needed, all needs to be tweaked and integrated to work together. dropping some rando library (even if compatible with the hardware) in the middle of that isn’t some plug and play affair.
Lastly, while a format or a standard is free to use, the algorithms and code can be subject to patents and additional licenses. In the case of HEIF, as I understand it, it’s just a container format. it is based HVEC, and that has a long list of patents behind it.
To quote this article