• 0 Posts
  • 12 Comments
Joined 1 year ago
cake
Cake day: June 17th, 2023

help-circle

  • That’s not what lossless data compression schemes do:
    In lossless compression the general idea is to create a codebook of commonly occuring patterns and use those as shorthand.
    For example, one of the simplest and now ancient algorithms LZW does the following:

    • Initialize the dictionary to contain all strings of length one.
    • Initialize the dictionary to contain all strings of length one.
    • Emit the dictionary index for W to output and remove W from the input.
    • Add W followed by the next symbol in the input to the dictionary.
    • repeat
      Basically, instead of rewriting long sequences, it just writes down the index into an existing dictionary of already seen sequences.

    However, once this is done, you now need to find an encoding that takes your characterset (the original characters+the new dictionary references) and turns it into bits.
    It turns out that we can do this optimally: Using an algorithm called Arithmetic coding we can align the length of a bitstring to the amount of information it contains.
    “Information” here meaning the statistical concept of information, which depends on the inverse likelihood a certain character is observed.
    Logically this makes sense:
    Let’s say you have a system that measures earthquakes. As one would expect, most of the time, let’s say 99% of the time, you will see “no earthquake”, while in 1% of the cases you will observe “earthquake”.
    Since “no earthquake” is a lot more common, the information gain is relatively small (if I told you “the system said no earthquake”, you could have guessed that with 99% confidence: not very surprising).
    However if I tell you “there is an earthquake” this is much more important and therefore is worth more information.

    From information theory (a branch of mathematics), we know that if we want to maximize the efficiency of our codec, we have to match the length of every character to its information content. Arithmetic coding now gives us a general way of doing this.

    However, we can do even better:
    Instead of just considering individual characters, we can also add in character pairs!
    Of course, it doesn’t make sense to add in every possible character pair, but for some of them it makes a ton of sense:
    For example, if we want to compress english text, we could give a separate codebook entry to the entire sequence “the” and save a ton of bits!
    To do this for pairs of characters in the english alphabet, we have to consider 26*26=676 combinations.
    We can still do that: just scan the text 600 times.
    With 3 character combinations it becomes a lot harder 26*26*26=17576 combinations.
    But with 4 characters its impossible: you already have half a million combinations!
    In reality, this is even worse, since you have way more than 26 characters: you have things like ", . ? ! and your codebook ids which blow up the size even more!

    So, how are we supposed to figure out which character pairs to combine and how many bits we should give them?
    We can try to predict it!
    This technique, called [PPM](Prediction by partial matching) is already very old (~1980s), but still used in many compression algorithms.
    The important trick is now that with deep learning, we can train even more efficient estimators, without loosing the lossless property:
    Remember, we only predict what things we want to combine, and how many bits we want to assign to them!
    The worst-case scenario is that your compression gets worse because the model predicts nonsensical character-combinations to store, but that never changes the actual information you store, just how close you can get to the optimal compression.

    The state-of-the-art in text compression already uses this for a long time (see Hutter Prize) it’s just now getting to a stage where systems become fast and accurate enough to also make the compression useful for other domains/general purpose compression.



  • No, it’s built into the protocol: think of it like as if every http request forces you to attach some tiny additional box containing the solution to a math puzzle.

    The twist is that you want the math puzzle to be easy to create and verify, but hard to compute. The harder the puzzle you solve, the more you get prioritized by the service that sent you the puzzle.

    If your puzzle is cheaper to create than hosting your service is, then it’s much harder to ddos you since attackers get stuck at the puzzle, rather than getting to your expensive service


  • Standard lossless compression (without further assumptions) is already very close to being as optimal as it can get: At some point the pure entropy of these huge datasets just is not containable anymore.

    The most likely savior in this case would be procedural rendering (i.e. instead of storing textures and meshes, you store a function that deterministically generates the meshes and textures). These already are starting to become popular due to better engine support, but pose a huge challenge from a design POV (the nice e.g. blender-esque interfaces don’t really translate well to this kind of process).



  • Not really: you have to keep in mind the amount of expertise and ressources that already went into silicon, as well as the geopolitics and sheer availability of silicon. The closest currently available competitor is probably gallium arsenide. That has a couple of disadvantages compared to silicon

    • It’s more expensive (both due to economies of scale and the fact that silicon is just much more abundant in general)
    • GaAs crystals are less stable, leading to smaller boules.
    • GaAs is a worse thermal conductor
    • GaAs has no native “oxide” (compare to SiO₂) which can be directly used as an insulator
    • GaAs mobilities are worse (Si is 500 vs GaAs 400), which means P channel FETs are naturally slower in GaAs, which makes CMOS structures impossible
    • GaAs is not a pure element, which means you get into trouble with mixing the elements
      You usually see GaAs combined with germanium substrates for solar panels, but rarely independently of that (GaAs is simply bad for logic circuits).
      In short: It’s not really useful for logic gates.

    Germanium itself is another potential candidate, especially since it can be alloyed with silicon which makes it interesting from an integration point-of-view.
    SiGe is very interesting from a logic POV considering its high forward and low reverse gain, which makes it interesting for low-current high-frequency applications. Since you naturally have heterojunctions which allow you to tune the band-gap (on the other hand you get the same problem as in GaAs: it’s not a pure element so you need to tune the band-gap).
    One problem specifically for mosfets is the fact that you don’t get stable silicon-germanium oxides, which means you can’t use the established silicon-on-insulator techniques.
    Cost is also a limiting factor: before even starting to grow crystals you have the pure material cost, which is roughly $10/kg for silicon, and $800/ kg for germanium.
    That’s why, despite the fact that the early semiconductors all relied on germanium, germanium based systems never really became practical: It’s harder to do mass production, and even if you can start mass production it will be very expensive (that’s why if you do see germanium based tech, it’s usually in low-production runs for high cost specialised components)

    There’s some research going on in commercialising these techniques but that’s still years away.




  • ZickZack@kbin.socialtoTechnology@lemmy.world*Permanently Deleted*
    link
    fedilink
    arrow-up
    28
    arrow-down
    2
    ·
    1 year ago

    They will make it open source, just tremendously complicated and expensive to comply with.
    In general, if you see a group proposing regulations, it’s usually to cement their own positions: e.g. openai is a frontrunner in ML for the masses, but doesn’t really have a technical edge against anyone else, therefore they run to congress to “please regulate us”.
    Regulatory compliance is always expensive and difficult, which means it favors people that already have money and systems running right now.

    There are so many ways this can be broken in intentional or unintentional ways. It’s also a great way to detect possible e.g. government critics to shut them down (e.g. if you are Chinese and everything is uniquely tagged to you: would you write about Tiananmen square?), or to get monopolies on (dis)information.
    This is not literally trying to force everyone to get a license for producing creative or factual work but it’s very close since you can easily discriminate against any creative or factual sources you find unwanted.

    In short, even if this is an absolutely flawless, perfect implementation of what they want to do, it will have catastrophic consequences.