I mean, think of it this way: it comes down to how often you come across words in any language including English (even in ENG: you may forget how to spell words correctly if you don’t use or encounter them often), kind of the same logic with Kanji: a Japanese person doesn’t know all Kanji in the same way English speakers doesn’t know every single word that exists in ENG.
There are over 5000 Kanji but only about half of that is used in Japanese or closer to 2136 while the remainder consist of ones only present within technical jargon (medicine, science, politics, etc.). or certain Kanji only has limited uses in some words (but mostly written in kana). That is also accounting for grammar being “straight forward” more than English or Euro languages.
The “real” hard part is numerous readings (depending whether it’s paired with kana or another kanji, reflected from kunyomi & onyomi plus nanori when applied in people’s names). What I hate about most online translators is that it often gets lost in translation (like words used in the wrong context but on their own it’s correct, however not right for the situation or topic at hand).
English is a fairly simple language that through the years has shaved off a lot of the complexities.
No grammatical genders - In languages like French, German or Spanish, you need to know the gender of the words, and there can be multiple, depending on the language, more than 3 genders.
Fairly simple declension - I refer you to this Wikipedika article to see all the ways you can have articles and pronouns be in German, depending on the grammatical case and gender of the word. And you need to use the correct one.
Fairly simple verb conjugation - Want 3rd person? Just add an “s”. Want past tense? Just add “ed”. That’s it basically for verb forms. Sure, there are irregular forms for past tense, but even those mostly follow a similar pattern, e.g “bought”, “fought”, “thought”, “caught”.
Behold this Wikipedia article again explaining all the verb forms you can have in German (look at those tables each containing a verb form lol). Oh btw, it does have irregular forms too.
The spelling is a mess, true, like, we don’t even have enough alphabets, and the way words are written are a complete mess compared to how they are spoken.
But comparing it to Kanji where you have learn roughly one symbol for each word? Where you literally can spend years learning it? Not learning the language itself, just learning how to read it. Oh, did I mention there are also 2 other alphabet sets that Japanese does have? Don’t worry though, that’s only around like a hundred or so additional symbols.
German certainly has a steep learning curve in the beginning, but I would argue it gets easier if you’re an advanced learner. Most more complex words are just compositions of easier words, pronunciation makes sense, the complex grammar quirks are either not used in everyday life or irrelevant (nobody cares if your say der, die, or das for any noun that’s not Nutella).
English on the other hand is easy to start but the learning curve never flattens. To pronounce a word correctly you often have to know the specific word beforehand or you’re lost (like with read, thyme, zealot, advertisement, …). To understand a new word you often have to look it up because compositional nouns are less common. That makes many new cool words but is less accessible.
Japanese Kanji are complicated. Ask a Chinese person learning Japanese, they will give you a good rant. Or ask a Japanese person who has been living abroad for a few years, they often forget many Kanji and have to relearn them. Main reason imho is that a lot of this has grown organically and the world has changed a lot over the past centuries, so many things would be done differently today.
As a native speaker, I have no frame of reference for English, but some of the people who learned it later indicated to me that it was hard to learn compared to other languages. Maybe it was because they went from one language to another similar language, but I’ve heard it from a few different people.