commit 43cf0e277d87867a3c57fcf22a6993228d8f35a9 Author: loki5512344 Date: Sun Sep 6 14:34:57 2026 +0200 chore: init monorepo with GPL-3.0 license, docs, backend skeleton, frontend wiring diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..3e0b177 --- /dev/null +++ b/.env.example @@ -0,0 +1,23 @@ +# Indexium — root .env.example (docker-compose + local dev) + +# --- Postgres (docker-compose service "postgres") --- +POSTGRES_DB=indexium +POSTGRES_USER=indexium +POSTGRES_PASSWORD=indexium + +# --- Backend --- +DATABASE_URL=postgres://indexium:indexium@postgres:5432/indexium +# локально без docker: postgres://indexium:indexium@localhost:5432/indexium +SERVER_PORT=8080 +RUST_LOG=info +REDIS_URL=redis://redis:6379 +# локально без docker: redis://localhost:6379 + +# --- Frontend (SvelteKit / Vite) --- +PUBLIC_API_URL=http://localhost:8080 +VITE_PUBLIC_API_URL=http://localhost:8080 + +# --- Optional / Phase 2 --- +# WEBHOOK_SECRET=change_me +# GITHUB_APP_ID= +# GITHUB_APP_PRIVATE_KEY= diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..6a8678a --- /dev/null +++ b/.gitignore @@ -0,0 +1,38 @@ +# Rust +/target/ +**/target/ +Cargo.lock + +# Node / SvelteKit +node_modules/ +.svelte-kit/ +build/ +.vite/ +dist/ +package-lock.json +bun.lockb + +# Env +.env +.env.local +.env.*.local + +# OS +.DS_Store +Thumbs.db + +# IDE +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# Logs +*.log +npm-debug.log* +bun-debug.log* + +# Misc +/tmp/ +coverage/ diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..53d1f3d --- /dev/null +++ b/LICENSE @@ -0,0 +1,675 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. + diff --git a/README.md b/README.md new file mode 100644 index 0000000..ad98954 --- /dev/null +++ b/README.md @@ -0,0 +1,74 @@ +# Indexium + +> Лёгкий, дешёвый и устойчивый к лимитам GitHub индексатор модов Minecraft. +> Бэкенд — **асинхронный событийный индексатор**: не хранит тяжёлые `.jar`, индексирует метаданные из GitHub Releases CDN и отдаёт быстрые JSON-ответы. + +## Архитектура (TL;DR) + +``` +GitHub --webhook release.published--> Ingestion API (HMAC check) --> Redis Queue --> Worker (Range-Request .jar → parse manifest) --> PostgreSQL --> Redis Cache --> Public REST API --> Web UI / Launchers +``` + +Подробно: [`docs/architecture.md`](docs/architecture.md) + +## Монорепо + +``` +/ +├── indexium-backend/ # Rust + Axum + sqlx + tokio +├── indexium-frontend/ # SvelteKit + TypeScript + Vite +├── docs/ # Архитектура, API, схема БД, ADR +├── todo.md # Роадмап по фазам +└── docker-compose.yml # (WIP) Postgres + Redis +``` + +Почему монорепо: см. [`docs/git-strategy.md`](docs/git-strategy.md) — один clone, атомарные изменения API+UI, один CI. + +## Быстрый старт (локально) + +```bash +# 1. Инфра +docker compose up -d # postgres + redis (когда будет compose файл) + +# 2. Бэкенд +cd indexium-backend +cp .env.example .env +sqlx migrate run +cargo run + +# 3. Фронт +cd ../indexium-frontend +bun install # или npm install +bun run dev +``` + +- Backend: `http://localhost:3000/api/v1/health` +- Frontend: `http://localhost:5173` + +## Стек + +| Слой | Технология | Зачем | +|------|-----------|-------| +| Backend Core | Rust (Axum) | Минимальный footprint, высокий throughput, async I/O | +| DB | PostgreSQL + FTS + pg_trgm | `JSONB`, полнотекстовый поиск без Elasticsearch, `pgvector` опционально | +| Cache & Queue | Redis / Valkey | Очередь (Streams) + L2-кэш популярных эндпоинтов | +| Worker | Rust tokio task | Парсинг `.jar` (zip-header stream), валидация, malware-чек | +| Auth | GitHub App / OAuth 2.0 | Только GitHub-вход, без локальных паролей | +| Frontend | SvelteKit | SSR, лёгкий бандл | + +## Документация + +- [`docs/architecture.md`](docs/architecture.md) — компоненты, lifecycle релиза, обход лимитов +- [`docs/database-schema.md`](docs/database-schema.md) — схема БД + индексы +- [`docs/api-spec.md`](docs/api-spec.md) — Public REST API v1 +- [`docs/deployment.md`](docs/deployment.md) — деплой, бэкапы +- [`docs/git-strategy.md`](docs/git-strategy.md) — почему монорепо и как работать с ним +- [`todo.md`](todo.md) — роадмап по фазам + +## Лицензия + +TBD + +## Контакты / Issues + +Используй GitHub Issues для багов и фич. Перед PR — `cargo fmt && cargo clippy`. diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..c8b85e0 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,55 @@ +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_DB: indexium + POSTGRES_USER: indexium + POSTGRES_PASSWORD: indexium + ports: + - "5432:5432" + volumes: + - pgdata:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U indexium -d indexium"] + interval: 5s + timeout: 5s + retries: 5 + start_period: 10s + + redis: + image: valkey/valkey:8-alpine + ports: + - "6379:6379" + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 5s + timeout: 3s + retries: 5 + + backend: + build: ./indexium-backend + environment: + DATABASE_URL: postgres://indexium:indexium@postgres:5432/indexium + SERVER_PORT: "8080" + RUST_LOG: info + REDIS_URL: redis://redis:6379 + ports: + - "8080:8080" + depends_on: + postgres: + condition: service_healthy + redis: + condition: service_healthy + + frontend: + build: ./indexium-frontend + environment: + PUBLIC_API_URL: http://localhost:8080 + VITE_PUBLIC_API_URL: http://localhost:8080 + ports: + - "5173:5173" + depends_on: + - backend + +volumes: + pgdata: diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..6d8559a --- /dev/null +++ b/docs/README.md @@ -0,0 +1,38 @@ +# docs — Индекс документации Indexium + +| Документ | Описание | +|----------|----------| +| [architecture.md](architecture.md) | Архитектура асинхронного событийного индексатора, компоненты, lifecycle, edge cases | +| [database-schema.md](database-schema.md) | Схема PostgreSQL, индексы FTS/pg_trgm, миграции | +| [api-spec.md](api-spec.md) | Public REST API v1, webhooks, auth, ошибки, rate limiting | +| [git-strategy.md](git-strategy.md) | Почему монорепо, workflow веток/коммитов | +| [deployment.md](deployment.md) | Деплой MVP на VPS, docker-compose, бэкапы, CI/CD | +| [adr/001-monorepo.md](adr/001-monorepo.md) | ADR-001: монорепо vs полирепо | +| [adr/002-async-indexer.md](adr/002-async-indexer.md) | ADR-002: асинхронный индексатор поверх GH Releases | +| [analytics.md](analytics.md) | bStats-аналог: SDK, ingestion, daily агрегаты, приватность | +| [adr/003-search-engine.md](adr/003-search-engine.md) | ADR-003: Postgres FTS + pg_trgm вместо Meilisearch | +| [adr/004-auth-strategy.md](adr/004-auth-strategy.md) | ADR-004: GitHub-only + PAT (+ Device Flow Phase 2) | +| [adr/005-analytics.md](adr/005-analytics.md) | ADR-005: bStats аналог — Postgres + daily_salt | + +## Как добавлять доки + +- Новые ADR: `docs/adr/NNN-kebab-title.md` по шаблону ниже. +- Диаграммы — Mermaid внутри markdown (рендерится в GitHub). + +### Шаблон ADR + +```markdown +# ADR-NNN: Заголовок + +Дата: 2026-09-06 +Статус: Принято | Отклонено | Отложено + +## Контекст +... + +## Решение +... + +## Последствия +... +``` diff --git a/docs/adr/001-monorepo.md b/docs/adr/001-monorepo.md new file mode 100644 index 0000000..82a8805 --- /dev/null +++ b/docs/adr/001-monorepo.md @@ -0,0 +1,26 @@ +# ADR-001: Монорепо vs Полирепо + +Дата: 2026-09-06 +Статус: Принято + +## Контекст +В корне два пакета: `indexium-backend` (Rust/Axum) и `indexium-frontend` (SvelteKit). Нужно решить как организовать git: один репозиторий на всё или два отдельных. Backend уже имел пустой `.git` без коммитов, фронт без гита. + +## Рассмотренные варианты +1. **Монорепо** — один `.git` в корне. +2. **Полирепо** — два независимых репозитория. +3. **Submodules** — корневой репо + сабмодули. + +## Решение +Выбрать **монорепо**. Удалить `indexium-backend/.git`, инициализировать `Indexium/.git` в корне. См. `docs/git-strategy.md`. + +Причины: атомарные изменения API+UI, один CI, проще onboarding, нет нужды в разных релизных каденсах на старте. + +## Последствия +- Положительные: один clone, один PR для кросс-пакетных изменений, один issue tracker. +- Отрицательные: при росте команды >10 может потребоваться разрезание (план через `git filter-repo`). +- Submodules отклонены из-за сложности DX. + +## Ссылки +- `docs/git-strategy.md` +- `todo.md` Phase 0 diff --git a/docs/adr/002-async-indexer.md b/docs/adr/002-async-indexer.md new file mode 100644 index 0000000..63c194a --- /dev/null +++ b/docs/adr/002-async-indexer.md @@ -0,0 +1,26 @@ +# ADR-002: Асинхронный событийный индексатор поверх GitHub Releases + +Дата: 2026-09-06 +Статус: Принято + +## Контекст +Нужен дешёвый и устойчивый к лимитам GitHub способ индексировать моды. Хранить `.jar` у себя дорого, проксировать трафик — упрёмся в bandwidth и rate limits. + +## Решение +Бэкенд не хранит артефакты. GitHub Releases CDN — источник правды для файлов. Мы только: +- принимаем webhook `release.published` (HMAC + queue + 202), +- воркер читает zip central directory через Range Request, +- парсит манифест (`fabric.mod.json` и т.д.), +- пишет метаданные в Postgres, +- отдаёт прямые `download_url` на `objects.githubusercontent.com`. + +Подробнее в `docs/architecture.md`. + +## Последствия +- Плюс: минимальный storage, нет egress costs, +5k–12.5k RPH через GitHub App. +- Минус: зависимость от доступности GitHub CDN (приемлемо — моды и так там). +- Вынесен malware-скан и SHA-256 сверка как обязательные. + +## Альтернативы +- Хранить файлы у себя (S3) — отклонено: дорого, дублирование. +- Полный pull `.jar` на каждый релиз — отклонено: трафик, медленно. diff --git a/docs/adr/003-search-engine.md b/docs/adr/003-search-engine.md new file mode 100644 index 0000000..e51145e --- /dev/null +++ b/docs/adr/003-search-engine.md @@ -0,0 +1,33 @@ +# ADR-003: Использование PostgreSQL FTS и pg_trgm вместо Meilisearch/Elasticsearch + +Дата: 2026-09-06 +Статус: Принято + +## Контекст +Для поиска модов по названию, описанию и авторам требуется полнотекстовый поиск и устойчивость к опечаткам (fuzzy search). Введение отдельного движка поиска (Meilisearch, OpenSearch, Elasticsearch) усложняет инфраструктуру и увеличивает потребление RAM (ещё один сервис в `docker-compose`, отдельный индекс, синхронизация). + +На MVP ожидается <50k модов, поисковый трафик <100 RPS, VPS за $5–10. + +## Решение +Использовать возможности PostgreSQL 16: +- `tsvector` + `GIN`-индексы для ранжированного полнотекстового поиска (`to_tsvector`, `ts_rank`, `plainto_tsquery`). +- Расширение `pg_trgm` для поиска с опечатками (Trigram Similarity, оператор `%`, `similarity()`). +- Материализованный `search_vector` с триггером на `INSERT/UPDATE` (см. `docs/database-schema.md`). + +Запрос: `search_vector @@ plainto_tsquery` + фильтр по `mod_versions` (`GIN (game_versions, loaders)`) + `ORDER BY ts_rank DESC` + fallback `similarity()` при 0 результатах. + +## Последствия + +- **Плюсы:** Нет дополнительных сервисов в `docker-compose`, экономия памяти (~0 доп. RAM vs +500MB–1GB у Meilisearch), атомарные транзакции при обновлении индекса (нет лагосинка), проще бэкапы. +- **Минусы:** При объёме >500k записей или >500 RPS скорость FTS в Postgres начинает уступать специализированным движкам (latency >100ms, нет typo-tolerance из коробки как в Meilisearch). +- **План миграции:** Если p95 latency поиска превысит 100ms (метрика в `tracing` + Grafana), вынести индекс в Meilisearch: добавить воркер-синк `mods` → Meilisearch, переключить `GET /mods` на Meilisearch с fallback на Postgres. Схема БД не меняется. + +## Альтернативы (отклонены на MVP) + +- **Meilisearch** — отличный typo-tolerance, но +1 сервис, нужен отдельный деплой и синк. +- **Elasticsearch / OpenSearch** — оверхед по RAM/диску, нужен кластер даже для малого объёма. +- **SQLite FTS** — не подходит, уже Postgres как основной. + +## Ссылки +- `docs/database-schema.md` — секция 2 (триггер `mods_search_vector_update`), секция 3 (примеры FTS + fuzzy). +- `docs/architecture.md` — секция 4 (Поиск без Elasticsearch). diff --git a/docs/adr/004-auth-strategy.md b/docs/adr/004-auth-strategy.md new file mode 100644 index 0000000..262ef21 --- /dev/null +++ b/docs/adr/004-auth-strategy.md @@ -0,0 +1,37 @@ +# ADR-004: Стратегия авторизации и профилей — GitHub-only + PAT (+ Device Flow позже) + +Дата: 2026-09-06 +Статус: Принято (MVP) / Запроектировано (Phase 2) + +## Контекст +Каталог open-source модов где 1 мод = 1 GitHub репо. Нужна минимальная, безопасная авторизация без паролей, но с поддержкой CLI/лаунчеров (Prism) и будущих Discord/коллекций. Пользователь предложил: GitHub App, PAT, Device Flow, Discord синк, дашборды, коллекции, геймификацию, SVG виджет. + +## Решение + +**MVP:** +- Вход только GitHub OAuth (`read:user`, `user:email`), JWT httpOnly cookie, без паролей. Читатели — anonymous. +- PAT с `SHA256` хранением и скоупами `read:mods|write:mods|webhooks:manage` для CI/лаунчеров. +- `verified` бейдж если репо публичное + лицензия + GitHub App установлен. +- Sponsors (GitHub/Patreon/Ko-fi) + star/follow (in-app) + SVG badges (`/v1/badges/:slug/downloads.svg`). + +**Phase 2 (спроектировано, не кодим сейчас):** +- Device Code Flow (RFC 8628) — Indexium как OAuth2 Provider для лаунчеров (`/oauth/device/code` → `/activate`). +- Discord linked_account + бот роли `Verified Modder`. +- Collections/Modlists с экспортом Prism/packwiz, Activity Feed, аналитика по версиям/лоадерам, PGP проверка. + +**Backlog:** краш-логи, лидерборды, Profile README. + +## Альтернативы +- Google/email логин — отклонён для MVP (публикация всё равно требует GitHub, лишняя сложность). +- Сразу Device Flow — отклонён ( +2 недели, PAT покрывает 80% кейсов). +- Discord как логин — отклонён (только linked). + +## Последствия +- Плюс: минимум GDPR, нет паролей, доказуемое владение репо, CLI готов через PAT. +- Минус: без GitHub аккаунта не опубликовать (осознанно, соответствует open-source философии). +- Миграция: таблицы `personal_access_tokens`, `linked_accounts`, `oauth_clients/device_codes` добавятся без breaking change. + +## Ссылки +- `docs/auth-profiles.md` §7-10 +- `docs/catalog-philosophy.md` +- `docs/api-spec.md` §Auth/Profiles diff --git a/docs/adr/005-analytics.md b/docs/adr/005-analytics.md new file mode 100644 index 0000000..25c826a --- /dev/null +++ b/docs/adr/005-analytics.md @@ -0,0 +1,29 @@ +# ADR-005: Собственный bStats-аналог для Indexium Analytics + +Дата: 2026-09-06 +Статус: Принято (дизайн) / К реализации в Phase 3 + +## Контекст +Скачивания накручиваются CI, нужна честная метрика популярности — активные установки в рантайме. bStats де-факто стандарт для Minecraft модов: lightweight SDK → POST gzip JSON → агрегация. Пользователь предложил полный дизайн с `server_uuid`, daily_salt, `mod_telemetry_pings` + `mod_daily_stats`, opt-out и сортировкой `active_servers`. + +## Решение +- **SDK:** MIT Java/Kotlin модуль `dev.indexium:analytics` ~15KB, `IndexiumMetrics(slug)` + `SimplePie`, уважает `-Dindexium.analytics.disable=true` и `config/indexium.json`. +- **Ingestion:** `POST /api/v1/analytics/submit` (gzip, без IP логов) → валидация allow-list → `server_hash = sha256(uuid + daily_salt)` → Redis 1/15мин → `mod_telemetry_pings` (TTL 30d). +- **Storage:** Postgres `mod_telemetry_pings` + `mod_daily_stats (breakdown_json)` + `analytics_salts`. Кроном `COUNT(DISTINCT server_hash)` раз в час. Хватает до 10M пингов/мес, далее TimescaleDB hypertable без смены схемы. +- **Serving:** `GET /mods/:slug/analytics?range=7d|30d|90d` (кэш 5м), `GET /badges/:slug/servers.svg`, `GET /mods?sort=active_servers`. +- **Приватность:** не храним IP, daily_salt ротация (не трекать сквозь дни), `custom_charts` ≤5 ключей, opt-out на клиенте. + +## Альтернативы +- Сторонний bStats.org — отклонён (внешняя зависимость, нет контроля, нет breakdown по нашим лоадерам). +- ClickHouse сразу — отклонён (оверхед для MVP, Postgres хватает). +- Хранить сырые пинги навсегда — отклонён (раздувание, достаточно daily агрегата). + +## Последствия +- Плюс: честная сортировка `active_servers`, графики для авторов, бейджи, без сторонних сервисов. +- Минус: +2 таблицы, крон-агрегация, SDK нужно публиковать в Maven Central. +- План: сначала Axum handler + агрегация, потом SDK (или наоборот — можно параллельно). + +## Ссылки +- `docs/analytics.md` +- `docs/api-spec.md` §Analytics +- `docs/database-schema.md` §3 diff --git a/docs/analytics.md b/docs/analytics.md new file mode 100644 index 0000000..2fe42a2 --- /dev/null +++ b/docs/analytics.md @@ -0,0 +1,119 @@ +# Indexium Analytics — собственный аналог bStats + +> Даём мододелам встроенную аналитику рантайма (активные серверы/клиенты, MC/Java/OS) без сторонних сервисов. Indexium получает честную метрику популярности — не по скачиваниям (накручиваются CI), а по реальным установкам. + +Основано на твоей схеме + правки под KISS/SOLID/приватность. + +--- + +## 1. Как работает bStats (база) + +1. **SDK в моде** — фоновый таймер каждые 30–60 мин собирает `mc_version, loader, java_version, os, player_count, server_uuid, custom_charts` → `POST` gzip JSON асинхронно, не блокируя главный поток. +2. **Ingestion** — бэкенд валидирует, rate-limit по `server_hash`, анонимизирует `server_uuid`. +3. **Aggregation** — сырые пинги → часовые/суточные агрегаты (Time Series), сырые удаляются по TTL. + +--- + +## 2. Indexium реализация + +### 2.1 Клиент — Lightweight Java/Kotlin модуль + +```java +// Fabric/NeoForge initialize() +IndexiumMetrics metrics = new IndexiumMetrics("sodium-extra", 12345); // slug + projectId (опц) +metrics.addCustomChart(new SimplePie("config_type", () -> config.getType())); +// respects: -Dindexium.analytics.disable=true, config/indexium.json { enabled: false } +``` + +**Что собираем (allow-list, ничего лишнего):** +- `mc_version` (1.20.1), `loader` (fabric/neoforge/forge/quilt), `loader_version` +- `java_version` (21.0.2), `os` (linux/windows/macos — без детальной версии), `arch` (x64/arm64) +- `player_count` (0 на клиенте, N на сервере), `server_uuid` (генерим раз, храним в `config/indexium-uuid.txt`) +- `mod_version` (из `fabric.mod.json`), `custom_charts` (String→String, до 5 ключей, до 32 символов) + +**Что НЕ собираем:** IP (не логируем), ник игрока, путь к файлам, список других модов (опционально по согласию, off по умолчанию). + +SDK: ~15KB, без зависимостей, `CompletableFuture` + `HttpURLConnection`, gzip. Лицензия MIT, публикуем в Maven Central как `dev.indexium:analytics:1.0.0`. + +### 2.2 API + +- `POST /api/v1/analytics/submit` — пинг от мода (gzip JSON, `Content-Encoding: gzip` опционально) +- `GET /api/v1/mods/:slug/analytics?range=7d|30d|90d` — графики для SvelteKit +- `GET /api/v1/badges/:slug/servers.svg` — бейдж активных серверов + +Пример payload (как в твоём ТЗ): +```json +{ + "mod_slug": "sodium-extra", + "server_uuid": "e8d9a0f1-4b2c-...", + "metrics": { + "mc_version": "1.20.1", + "loader": "fabric", + "java_version": "21.0.2", + "os": "Linux", + "player_count": 12, + "custom_charts": { "gui_theme": "dark" } + } +} +``` + +### 2.3 Хранение — PostgreSQL (MVP) → TimescaleDB/ClickHouse при росте + +На MVP хватает Postgres + daily агрегат (как ты предложил). Сырые пинги храним 30 дней, агрегаты — навсегда. + +```sql +-- Полуагрегат: один пинг = одна строка, TTL 30 дней через cron +CREATE TABLE mod_telemetry_pings ( + id BIGSERIAL PRIMARY KEY, + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + server_hash CHAR(64) NOT NULL, -- sha256(server_uuid + daily_salt) + mc_version VARCHAR(16) NOT NULL, + loader VARCHAR(16) NOT NULL, + os VARCHAR(16) NOT NULL, + java_version VARCHAR(16) NOT NULL, + player_count INT NOT NULL DEFAULT 0, + pinged_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); +CREATE INDEX idx_telemetry_lookup ON mod_telemetry_pings (mod_id, pinged_at DESC); +CREATE INDEX idx_telemetry_hash ON mod_telemetry_pings (server_hash, pinged_at); + +-- Суточный агрегат (хранится навсегда) +CREATE TABLE mod_daily_stats ( + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + date DATE NOT NULL, + active_servers INT NOT NULL DEFAULT 0, -- COUNT(DISTINCT server_hash) + active_players INT NOT NULL DEFAULT 0, -- SUM(player_count) по последним пингам сервера за день + breakdown_json JSONB NOT NULL, -- { mc_versions:{}, loaders:{}, os:{}, java:{}, custom:{gui_theme:{dark: 120}} } + PRIMARY KEY (mod_id, date) +); +``` + +**Агрегация:** воркер-кроном раз в час: `INSERT INTO mod_daily_stats ... ON CONFLICT DO UPDATE` группировкой по `server_hash` (последний пинг сервера за день). Через `pg_cron` или tokio `interval` в бэкенде. + +**Масштаб:** при >10M пингов/мес — мигрируем на TimescaleDB hypertable (`create_hypertable('mod_telemetry_pings','pinged_at')`) или ClickHouse. Схема не меняется. + +### 2.4 Защита и анонимность (критично) + +- **Хеш + daily_salt:** `server_hash = sha256(server_uuid + salt_for_today)`. Соль ротируется в `analytics_salts(date, salt)`, храним 2 дня. Нельзя трекать сервер сквозь дни, но можно считать уникальные за день. +- **Не храним IP:** `tower_http::TraceLayer` без IP, `X-Forwarded-For` игнорируем, в логах — `/analytics/submit 200` без IP. +- **Opt-Out:** SDK проверяет в порядке: JVM флаг `-Dindexium.analytics.disable=true` → `global_privacy.json` (`.minecraft/config/indexium.json { enabled:false }`) → `config//indexium.json`. Если любой `false` — не шлём. +- **Rate limit:** Redis `SET server_hash:mod_slug NX EX 900` — 1 пинг / 15 мин. Ответ `429` с `Retry-After`, SDK бэкофф 30 мин. +- **Валидация:** `mod_slug` должен существовать, `mc_version`/`loader` из allow-list, `custom_charts` ≤5 ключей, `player_count` 0–10000. Иначе `400`. + +--- + +## 3. Фичи для профиля и карточки мода + +1. **Live Charts (SvelteKit + LayerChart/Chart.js):** `GET /mods/:slug/analytics?range=30d` → `{ daily: [{date, active_servers, active_players}], breakdown: {mc_versions, loaders, os} }`. Графики: активные серверы (линия), разбивка по MC (пончик), лоадерам (бар). +2. **Badge:** `https://api.indexium.example.com/v1/badges/sodium-extra/servers.svg` → `Active Servers: 1.2k` (из `mod_daily_stats` за вчера, кэш 1h). +3. **Сортировка "Real-world Usage":** `GET /mods?sort=active_servers` — `ORDER BY (SELECT active_servers FROM mod_daily_stats WHERE date = CURRENT_DATE -1)`, а не по скачиваниям. Фильтр против накрутки CI. + +--- + +## 4. Что не делаем (чтобы не стать spyware) + +- Не собираем ник, чат, координаты, список всех модов без явного согласия (если включим — отдельный `custom_charts` с opt-in). +- Не fingerprint'им по железу. +- SDK открыт (MIT) — любой может проверить что шлём (как bStats — код на GitHub). + +См. `adr/005-analytics.md`, `api-spec.md` §Analytics, `database-schema.md` §telemetry. diff --git a/docs/api-spec.md b/docs/api-spec.md new file mode 100644 index 0000000..c6cddef --- /dev/null +++ b/docs/api-spec.md @@ -0,0 +1,318 @@ +# Public REST API — Indexium v1 + +Base URL: `https://api.indexium.example.com/api/v1` (локально `http://localhost:3000/api/v1`) + +Все ответы — `application/json`. Пагинация — `page`/`limit` (MVP) → cursor позже. Кэш — `Cache-Control: public, max-age=60`, `ETag`. + +--- + +## Health + +### `GET /health` +Проверка живости + БД + Redis. + +**200** +```json +{ "status": "ok", "db": "up", "redis": "up", "version": "0.1.0" } +``` + +--- + +## Webhooks (internal) + +### `POST /webhooks/github` +Принимает GitHub Webhook `release`. + +Headers: +- `X-GitHub-Delivery: uuid` +- `X-Hub-Signature-256: sha256=...` +- `X-GitHub-Event: release` + +Body: raw JSON от GitHub. + +**202** — принято в очередь +```json +{ "status": "accepted", "delivery_id": "..." } +``` +**401** — неверная подпись +**409** — уже обработано (идемпотентность) + +Логика: HMAC проверка → дедуп по `delivery_id` → push в Redis Streams → 202. + +--- + +## Mods + +### `GET /mods` + +Query params: + +| param | type | описание | +|-------|------|----------| +| `query` | string | FTS по name/summary/README | +| `gameVersion` | string | фильтр `1.20.1` | +| `loader` | string | `fabric` \| `quilt` \| `neoforge` \| `forge` | +| `page` | int | default 1 | +| `limit` | int | default 20, max 50 | +| `sort` | string | `relevance` \| `newest` \| `popular` \| `active_servers` | + +**200** +```json +{ + "data": [ + { + "slug": "sodium-extra", + "name": "Sodium Extra", + "summary": "Extra optimizations", + "author": "flashy", + "icon_url": "https://...", + "game_versions": ["1.20.1"], + "loaders": ["fabric"], + "latest_version": "1.2.3", + "download_url": "https://github.com/.../releases/download/...", + "updated_at": "2026-09-01T12:00:00Z" + } + ], + "pagination": { "page": 1, "limit": 20, "total": 142, "pages": 8 } +} +``` + +Кэшируется в Redis по ключу `mods:query=...:gv=...:loader=...:page=...` TTL 60s. + +### `GET /mods/:slug` + +**200** +```json +{ + "slug": "sodium-extra", + "name": "Sodium Extra", + "summary": "...", + "description": "... (markdown)", + "github_repo": "owner/repo", + "author": { "login": "flashy", "avatar_url": "https://..." }, + "icon_url": "https://...", + "verified": true, + "versions": [ + { + "version_number": "1.2.3", + "game_versions": ["1.20.1"], + "loaders": ["fabric"], + "download_url": "https://github.com/.../releases/download/v1.2.3/sodium-extra-1.2.3.jar", + "file_sha256": "abc...", + "file_size": 123456, + "published_at": "2026-09-01T12:00:00Z" + } + ] +} +``` +**404** — `{"error":"mod_not_found"}` + +### `GET /mods/:slug/icon` + +Отдаёт иконку мода. Воркер при индексации извлекает `assets//icon.png` (или `icon` из `fabric.mod.json` → путь внутри jar) → сохраняет в кэш/проксирует. + +- **200** — `image/png` / `image/webp` с `Cache-Control: public, max-age=86400`, `ETag`. Если иконки нет → `302` на `raw.githubusercontent.com` fallback или дефолтная заглушка. +- **404** — мод не найден. + +> Альтернатива на MVP: не хранить иконку у себя, а отдавать `icon_url` как прямую ссылку `https://raw.githubusercontent.com////src/main/resources/assets/...`. Эндпоинт `/icon` тогда — 302 редирект + кэш заголовков. + +### `POST /mods/resolve` — пакетный резолв для лаунчеров + +Принимает список модов + окружение, возвращает дерево прямых скачиваний и зависимостей (для Prism / Modrinth-compatible клиентов). + +**Request** +```json +{ + "game_version": "1.20.1", + "loader": "fabric", + "mods": [ + { "slug": "sodium-extra", "version": "1.2.3" }, + { "slug": "lithium", "version": "latest" } + ] +} +``` + +**200** +```json +{ + "resolved": [ + { + "slug": "sodium-extra", + "version_number": "1.2.3", + "download_url": "https://github.com/.../releases/download/.../sodium-extra-1.2.3.jar", + "file_sha256": "abc...", + "file_size": 123456, + "dependencies": [{ "slug": "sodium", "version_range": ">=0.5.0", "resolved_version": "0.5.8" }] + }, + { + "slug": "sodium", + "version_number": "0.5.8", + "download_url": "https://github.com/.../sodium-0.5.8.jar", + "file_sha256": "def...", + "file_size": 654321, + "dependencies": [] + } + ], + "unresolved": [] +} +``` + +- `version: "latest"` → резолвит последнюю совместимую с `game_version` + `loader`. +- Транзитивные зависимости резолвятся рекурсивно (BFS, max depth 20, защита от циклов). +- **422** — несовместимая комбинация `game_version`/`loader`. +- Кэшируется по ключу `resolve:gv:loader:hash(mods)` TTL 60s. + +### `GET /mods/:slug/versions/:version` + +Детали конкретной версии. Аналогично элементу массива выше + зависимости: + +```json +{ + "mod_slug": "sodium-extra", + "version_number": "1.2.3", + "game_versions": ["1.20.1"], + "loaders": ["fabric"], + "dependencies": [{ "mod_id": "sodium", "version_range": ">=0.5.0" }], + "download_url": "https://github.com/...", + "file_sha256": "...", + "file_size": 123456, + "published_at": "..." +} +``` + +### `POST /mods/import` (auth required) + +Импорт репозитория по GitHub OAuth. + +Headers: `Authorization: Bearer ` + +Body: +```json +{ "repo": "owner/repo" } +``` + +Логика: проверить что токен имеет доступ к репо → fetch `fabric.mod.json` из default branch → создать запись `mods` → повесить webhook. + +**201** — создан +**409** — уже импортирован +**422** — манифест не найден + +--- + +## Auth + +### `GET /auth/github` → 302 redirect на GitHub OAuth +### `GET /auth/github/callback?code=...` → обмен code→token, установка httpOnly cookie / JWT + +### `POST /auth/tokens` (auth) — PAT creation +Body: `{ "name": "ci-token", "scopes": ["read:mods","write:mods"], "expires_in_days": 30 }` → `201 { token: "idx_...", id, expires_at }` (токен показывается 1 раз, храним hash). `Authorization: Bearer idx_...` для API. + +### `GET /auth/tokens` / `DELETE /auth/tokens/:id` — список/отзыв. + +### Device Flow (Phase 2, RFC 8628) +- `POST /oauth/device/code` → `{ device_code, user_code: "ABCD-1234", verification_uri: "https://indexium.example.com/activate", expires_in: 600 }` +- `GET /activate` (frontend) — ввод `user_code` → consent → `POST /oauth/device/verify { user_code }` +- `POST /oauth/token` grant_type=`urn:ietf:params:oauth:grant-type:device_code` → `{ access_token, refresh_token }` +- Лаунчер поллит `/oauth/token` до получения токена. + +### `GET /auth/discord` → линк Discord (linked_account, не логин). `GET /auth/discord/callback` → запись в `linked_accounts`. + +## Profiles & Social + +### `GET /u/:login` / `GET /org/:login` — публичный профиль (кэш 60s, ISR) +### `POST /mods/:slug/star` / `DELETE /mods/:slug/star` — звезда (auth) +### `POST /u/:login/follow` / `DELETE /u/:login/follow` — подписка на автора с опционально `?game_version=1.20.1&loader=fabric` +### `GET /collections` / `POST /collections` (auth, body: `{ title, description, mods: [{slug, version}] }`) +### `GET /collections/:slug` / `GET /collections/:slug/export?format=prism|packwiz` +### `GET /v1/badges/:slug/downloads.svg` / `GET /v1/badges/:slug/version.svg` — SVG виджет для README (public, кэш 1h) + +## Analytics (bStats аналог, см. docs/analytics.md) + +### `POST /api/v1/analytics/submit` — пинг от мода (gzip опционально) +Headers: `Content-Type: application/json`, `Content-Encoding: gzip` (optional) +Body: +```json +{ + "mod_slug": "sodium-extra", + "server_uuid": "e8d9a0f1-4b2c-...", + "metrics": { + "mc_version": "1.20.1", + "loader": "fabric", + "java_version": "21.0.2", + "os": "Linux", + "player_count": 12, + "custom_charts": { "gui_theme": "dark" } + } +} +``` +- Валидация: `mod_slug` exists, allow-list версий/лоадеров, `custom_charts` ≤5 ключей. +- Анонимизация: `server_hash = sha256(server_uuid + daily_salt)` — IP не храним. +- Rate limit: 1 пинг / 15 мин per `server_hash+mod_slug` (Redis `SET NX EX 900`) → `429`. +- Opt-Out: respect `-Dindexium.analytics.disable=true` на клиенте. +- **200** `{ "status": "ok" }` **400** validation **429** rate_limited + +### `GET /api/v1/mods/:slug/analytics?range=7d|30d|90d` +Public, кэш `public, max-age=300`. +```json +{ + "mod_slug": "sodium-extra", + "range": "30d", + "daily": [ + { "date": "2026-09-01", "active_servers": 450, "active_players": 3200 }, + { "date": "2026-09-02", "active_servers": 470, "active_players": 3400 } + ], + "breakdown": { + "mc_versions": { "1.20.1": 400, "1.21": 50 }, + "loaders": { "fabric": 420, "neoforge": 30 }, + "os": { "Linux": 200, "Windows": 250 }, + "java": { "21": 300, "17": 150 }, + "custom": { "gui_theme": { "dark": 120, "light": 30 } } + } +} +``` +**404** mod_not_found. Источник: `mod_daily_stats`. + +### `GET /api/v1/badges/:slug/servers.svg` — бейдж активных серверов (как downloads.svg, кэш 1h) +SVG `Active Servers: 1.2k` из `mod_daily_stats` за вчера. + +## Sponsors & Badges + +- `GET /u/:login` отдаёт `sponsors: { github, patreon, kofi, bmc }` и `badges: ["early_adopter","verified"]` +- Бейджи выдаются воркером (`badges` таблица), SVG — динамически. + +--- + +## Ошибки + +Единый формат: + +```json +{ "error": "validation_error", "message": "gameVersion must be semver", "details": {...} } +``` + +Коды: +- `400` validation_error +- `401` unauthorized +- `404` not_found +- `429` rate_limited (headers `Retry-After`) +- `500` internal_error + +--- + +## Rate limiting + +- Public API: 60 req/min per IP (Redis). +- Webhook: без лимита, но HMAC обязателен. + +--- + +## Версионирование + +- URL версионирование `/api/v1`. +- Breaking changes → `/api/v2` + 6 мес поддержка v1. + +--- + +## OpenAPI + +Спека будет жить в `indexium-backend/openapi.yaml` (генерировать из Axum через `utoipa` когда созреет). На MVP — этот markdown как источник правды. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..308e65e --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,219 @@ +# Архитектура Indexium — Асинхронный событийный индексатор + +> Цель: сделать сервис максимально лёгким, дешёвым в обслуживании и устойчивым к ограничениям GitHub. +> Принцип: бэкенд **не хранит** тяжёлые файлы — артефакты отдаются с GitHub Releases CDN, мы валидируем, индексируем метаданные и выдаём быстрые JSON-ответы. + +--- + +## 1. Обзор системных компонентов + +``` + ┌───────────────────────────────────────────────┐ + │ GitHub Ecosystem │ + └───────┬───────────────────────────────▲───────┘ + │ │ + │ 1. Webhook (release.published)│ 4. Read Assets / Metadata + ▼ │ +┌─────────────────────────────────────────────────────────┴────────────────────────────────┐ +│ Ваш Бэкенд (Event-Driven Indexer) │ +│ │ +│ ┌───────────────────────┐ ┌──────────────────────┐ ┌─────────────────────┐ │ +│ │ Webhook Ingestion API │ ────> │ Async Job Queue │ ────> │ Worker / Indexer │ │ +│ │ (Fast Signature Check)│ │ (Redis / NATS) │ │ (Jar Parser & AST) │ │ +│ └───────────────────────┘ └──────────────────────┘ └──────────┬──────────┘ │ +│ │ │ +│ ▼ │ +│ ┌───────────────────────┐ ┌──────────────────────┐ ┌─────────────────────┐ │ +│ │ Edge Public REST/v1 │ <──── │ Read-Heavy Cache │ <──── │ Relational DB │ │ +│ │ (High Throughput API) │ │ (Redis Key-Value) │ │ (PostgreSQL + FTS) │ │ +│ └───────────────────────┘ └──────────────────────┘ └─────────────────────┘ │ +└─────────────────────────────────────────────▲────────────────────────────────────────────┘ + │ + │ 5. Query Mods / Index + │ + ┌───────────┴───────────┐ + │ Web UI / Launchers │ + └───────────────────────┘ +``` + +### Потоки данных +1. **Ingestion** — GitHub шлёт webhook → Ingestion API валидирует HMAC → кладёт job в очередь → отвечает `202`. +2. **Indexing** — Worker читает job → делает `Range Request` к `.jar` → парсит манифест → пишет в Postgres → инвалидирует кэш. +3. **Serving** — Лаунчер / Web UI дергает `GET /api/v1/mods` → читаем Redis → miss → Postgres FTS → кэшируем 60s → отдаём JSON. + +--- + +## 2. Выбор технологического стека + +| Слой | Технология | Почему именно это? | +| --- | --- | --- | +| **Backend Core** | **Rust (Axum)** | Минимальный memory footprint, высокий throughput, быстрый async I/O при парсинге. Альтернатива Go (Fiber) — допустима на Phase 2. | +| **Main Database** | **PostgreSQL** | Нативный FTS, `JSONB` для зависимостей, `pg_trgm` для fuzzy, `pgvector` опционально для семантики. | +| **Cache & Queue** | **Redis / Valkey** | Одновременно брокер очередей (Streams/PubSub) и L2-кэш популярных эндпоинтов. | +| **Worker Engine** | **Rust Background Worker (tokio)** | Потребляет webhook-события, скачивает только zip-header stream, валидирует байткод. | +| **Auth & Security** | **GitHub App / OAuth 2.0** | Вход только через GitHub, без локальных паролей. | +| **Frontend** | **SvelteKit + TypeScript** | SSR, маленький бандл, быстрый dev. | + +> Обоснование против Elasticsearch/Meilisearch на старте: `to_tsvector` + `pg_trgm` переваривают десятки тысяч модов с <5ms без отдельного кластера. Миграция на внешний поиск — только если FTS упрётся. + +--- + +## 3. Жизненный цикл релиза (Release Lifecycle) + +### 3.1 Регистрация мода (Onboarding) +1. Разработчик логинится через GitHub OAuth. +2. Жмёт «Импортировать репозиторий» → сервис проверяет наличие манифеста (`fabric.mod.json`, `neoforge.mods.toml`, `quilt.mod.json`) в default branch. +3. Устанавливается GitHub App + Webhook на события `release` и `push`. + +### 3.2 Обработка webhook (Ingestion & Job Dispatch) +1. GitHub шлёт `release.published`. +2. **Ingestion API** валидирует `X-Hub-Signature-256` (HMAC SHA-256 с `WEBHOOK_SECRET`), проверяет идемпотентность по `X-GitHub-Delivery`. +3. Кладёт задачу в Redis Queue, отвечает `202 Accepted` за <50ms (чтобы не висеть по таймауту GitHub — 10s). + +### 3.3 Работа воркера-индексатора (Worker Execution) — детальный алгоритм `jar_parser.rs` + +> Цель: не скачивать весь `.jar` (может быть 20–50MB), а прочитать только нужный манифест через 2–3 Range-запроса. + +``` +download_url = "https://github.com/owner/repo/releases/download/v1.2.3/mod-1.2.3.jar" + │ + Step 1: HEAD ─────┤ + ▼ + Content-Length: 12345678 + Accept-Ranges: bytes + (если нет Content-Length → GET Range: bytes=0-0 + парс Content-Range) + │ + Step 2: GET tail ─┤ Range: bytes=-65536 (последние 64KB) + ▼ + Найти EOCD (End of Central Directory) = 0x06054b50 + Из EOCD: central_dir_offset, central_dir_size, num_entries + │ + Step 3: GET central dir ─┤ Range: bytes=central_dir_offset..central_dir_offset+size + ▼ + Парс Central Directory headers (0x02014b50) + Найти entry: fabric.mod.json | quilt.mod.json | neoforge.mods.toml | mcmod.info + + icon (assets//icon.png если указан в манифесте) + → получить local_header_offset, compressed_size + │ + Step 4: GET manifest ─┤ Range: bytes=local_header_offset.. + compressed_size + header + ▼ + Распаковать (DEFLATE/STORE), парс JSON/TOML, валидация + + SHA-256 сверка, file_size = Content-Length +``` + +**Детали реализации:** +1. `HEAD` — обязателен, чтобы получить `Content-Length` и убедиться `Accept-Ranges: bytes`. Таймаут 5s, retry 2. +2. Последние 64KB достаточно для EOCD даже для jar с 10k файлов (EOCD в конце). Если не найден — fallback к последним 128KB. +3. Central Directory читается одним запросом (обычно 5–30KB). Парсим `central_dir_offset/size` из EOCD. +4. Манифест — 4-й запрос только если нужен (часто 1–3KB). Иконка — опционально 5-й запрос, кэшируется и отдаётся через `GET /mods/:slug/icon`. +5. Все `GET` — `reqwest` с `header("Range", ...)`, проверка `206 Partial Content`, иначе fallback к полному скачиванию с лимитом 10MB. + +**Ошибки:** `412` если `Accept-Ranges` != bytes → full download; `404` на Range → retry без Range; повреждённый ZIP → помечаем `suspicious` и DLQ. + +Код: `indexium-backend/src/worker/jar_parser.rs` — чистые функции `find_eocd()`, `parse_central_dir()`, `fetch_manifest()` без I/O в тестах. + +### 3.4 Агрегация и индексация (Storage & Cache Invalidation) +1. Сохраняет версию в `mod_versions` (см. `database-schema.md`). +2. `download_url` = прямая ссылка `https://github.com/.../releases/download/...` (CDN `objects.githubusercontent.com`). +3. Инвалидирует / обновляет Redis-кэш для поиска и карточки мода. + +--- + +## 4. Обход ключевых ограничений (Edge Cases) + +### GitHub Rate Limits +- Запросы воркеров — от имени **GitHub App Installs** (5k–12.5k RPH на инсталл vs 60 RPH анонимных). +- Скачивание не проксируем — выдаём клиентам прямые CDN-ссылки, трафик не идёт через нас. + +### Безопасность (Malware Protection) +- Сверка `SHA-256` ассета с `.sha256` если есть. +- Сканирование байткода: флаг на `Runtime.getRuntime().exec()`, `ProcessBuilder`, `URLClassLoader`, сетевые вызовы в `` / `FabricModInitializer`. +- Карантин: помечаем версию `suspicious = true`, не показываем в публичном поиске до ручной проверки. + +### Поиск без Elasticsearch +- Postgres `to_tsvector('russian'|'english', name || summary)` + `GIN`. +- `pg_trgm` (`similarity()`, `%` оператор) для опечаток. +- Материализованный `search_vector` + триггер на update. + +### Надёжность очереди +- Retry с exponential backoff (3 попытки), DLQ (dead-letter) для ручного разбора. +- Идемпотентный воркер: `ON CONFLICT (mod_id, version_number) DO UPDATE`. + +### Телеметрия (bStats аналог, см. docs/analytics.md) +- **Ingestion:** `POST /api/v1/analytics/submit` (gzip JSON, без IP логов) → валидация allow-list → `server_hash = sha256(uuid + daily_salt)` → Redis rate limit 1/15мин → `INSERT mod_telemetry_pings`. +- **Aggregation:** кроном раз в час `COUNT(DISTINCT server_hash)` + `breakdown_json` → `mod_daily_stats`, TTL 30 дней для сырых пингов (`DELETE WHERE pinged_at < NOW()-30d`). +- **Serving:** `GET /mods/:slug/analytics?range=30d` (кэш 5 мин) + `GET /badges/:slug/servers.svg` + `sort=active_servers` для честной сортировки по реальным установкам, а не накрученным скачиваниям. +- **Масштаб:** Postgres хватает до 10M пингов/мес, далее TimescaleDB hypertable без смены схемы. + +--- + +## 5. Схема структуры БД (Core Entity Relation) + +См. детально в [`database-schema.md`](database-schema.md). Коротко: + +```sql +-- Таблица модов +CREATE TABLE mods ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + github_repo_id BIGINT UNIQUE NOT NULL, + slug VARCHAR(64) UNIQUE NOT NULL, + name VARCHAR(128) NOT NULL, + summary TEXT, + author_github_id BIGINT NOT NULL, + default_branch VARCHAR(32) DEFAULT 'main', + created_at TIMESTAMPTZ DEFAULT now() +); + +-- Таблица версий (релизов) +CREATE TABLE mod_versions ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + mod_id UUID REFERENCES mods(id) ON DELETE CASCADE, + version_number VARCHAR(32) NOT NULL, + game_versions VARCHAR(32)[] NOT NULL, -- e.g. ['1.20.1', '1.20.2'] + loaders VARCHAR(16)[] NOT NULL, -- e.g. ['fabric', 'quilt'] + download_url TEXT NOT NULL, -- GitHub Release Direct Asset URL + file_sha256 CHAR(64) NOT NULL, + published_at TIMESTAMPTZ NOT NULL, + UNIQUE(mod_id, version_number) +); + +CREATE INDEX idx_versions_lookup ON mod_versions USING GIN (game_versions, loaders); +``` + +Дополнительно: `authors`, `webhook_deliveries` (идемпотентность), `search_vector`, `mod_telemetry_pings`/`mod_daily_stats` (аналитика). + +--- + +## 6. Масштабирование и эволюция + +| Этап | Нагрузка | Действие | +|------|----------|----------| +| MVP | <10k модов, <100 RPS | Один инстанс Axum + Postgres + Redis, воркер в том же бинаре (tokio spawn) | +| Growth | 10k-100k модов, 1k RPS | Вынос воркера в отдельный деплой, реплика Postgres RO, Redis Cluster | +| Scale | 100k+ модов, 10k RPS | NATS JetStream вместо Redis Streams, read-replica + шардирование по `slug`, CDN перед API (Cloudflare) | + +--- + +## 7. Нефункциональные требования + +- **Latency**: `GET /mods` p95 < 80ms (cache hit), < 200ms (cache miss + FTS). +- **Availability**: 99.9% (допустим 43 мин downtime/мес на MVP). +- **Cost**: < $20/мес на MVP (1 VPS Hetzner + managed Postgres free tier). +- **Security**: HMAC, GitHub App, no local passwords, malware scan. + +--- + +## 8. Диаграмма деплоя (MVP) + +``` +[GitHub] --webhook--> [Axum Ingestion :3000] --> [Redis :6379] --> [Worker] + | | + v v + [Postgres :5432] <--+ + | +[Browser/Launcher] --> [Axum Public API :3000] --> [Redis Cache] + \ + --> [SvelteKit :5173] (SSR, fetch API) +``` + +Все сервисы — `docker compose` локально, один VPS в проде. diff --git a/docs/auth-profiles.md b/docs/auth-profiles.md new file mode 100644 index 0000000..4938966 --- /dev/null +++ b/docs/auth-profiles.md @@ -0,0 +1,232 @@ +# Профили, аккаунты и авторизация — дизайн Indexium + +> Цель: максимально лёгкая, но крутая система профилей без паролей, где GitHub — источник правды. + +--- + +## 1. TL;DR — рекомендуем для MVP + +**Авторизация: только GitHub OAuth / GitHub App.** Никаких паролей, email+пароль, Google и т.д. на старте. + +**Почему именно GitHub-only:** + +| Плюс | Минус | +|---|---| +| 1 клик, нет форм регистрации | Отсекаем тех у кого нет GitHub (но они и моды не публикуют) | +| Доказуемое владение репозиторием (`GET /repos` с токеном) | Зависимость от GitHub OAuth (но у нас и так всё на GitHub) | +| Аватар, ник, био подтягиваются автоматически | Нет anon-публикаций (и это хорошо для open source) | +| Один токен — и публикация, и вебхуки, и профиль | Если GitHub лежит — логин не работает (редкость) | +| Нет хранения паролей, нет утечек | | + +> **Вывод:** для каталога где `1 мод = 1 GitHub репо` — GitHub-only это не ограничение, а фича. Пользователи-читатели (игроки) могут смотреть каталог **без логина вообще**. Логин нужен только авторам. + +--- + +## 2. Роли и модель аккаунта + +### Роли + +- **Reader (anonymous)** — ищет, качает по прямым ссылкам, смотрит профили. Без аккаунта. +- **Author** — залогинен через GitHub, импортировал хотя бы один репо. Может публиковать релизы (через `git push` + webhook, без кнопки "загрузить jar"). +- **Contributor** — указан в `mod_authors` с `role=contributor`, не обязательно owner репо. Получает бейдж на карточке мода. +- **Moderator / Admin** — ручная выдача, может ставить `verified` / `suspicious`, банить. + +### Что храним (минимум GDPR) + +```sql +-- уже есть authors, расширяем (см. database-schema.md) +CREATE TABLE authors ( + github_id BIGINT PRIMARY KEY, + login VARCHAR(39) NOT NULL UNIQUE, -- github login + display_name VARCHAR(128), -- from GitHub name + avatar_url TEXT, + bio TEXT, -- from GitHub bio (кэш, обновляем раз в день) + company VARCHAR(128), + location VARCHAR(128), + website TEXT, -- blog + created_at TIMESTAMPTZ DEFAULT now(), + last_synced_at TIMESTAMPTZ +); + +CREATE TABLE mod_authors ( + mod_id UUID REFERENCES mods(id) ON DELETE CASCADE, + github_id BIGINT REFERENCES authors(github_id) ON DELETE CASCADE, + role VARCHAR(16) NOT NULL DEFAULT 'owner', -- owner | maintainer | contributor + PRIMARY KEY (mod_id, github_id) +); +``` + +Никаких email в открытом виде (берём только для JWT, не показываем), никаких паролей. `github_id` — неизменяемый PK, `login` может смениться — обновляем по webhook `user.renamed` или при следующем логине. + +### Сессии + +- **JWT (httpOnly cookie)**: `sub: github_id`, `login`, `exp: 7d`. Подпись `HS256` с `JWT_SECRET` или `RS256` если хотим ротацию. +- **Не храним сессии в Redis на MVP** — stateless JWT достаточно. Позже — refresh token в `author_sessions`. +- **CSRF**: `SameSite=Lax` + `Origin` check для `POST /mods/import`. + +--- + +## 3. Флоу авторизации (GitHub OAuth) + +``` +[User] → GET /auth/github → 302 https://github.com/login/oauth/authorize?client_id=...&scope=read:user,repo + → GitHub login → 302 /auth/github/callback?code=... + → Backend: POST https://github.com/login/oauth/access_token (code → access_token) + → GET https://api.github.com/user (с токеном) → { id, login, avatar_url, name, bio } + → UPSERT authors + → Set-Cookie: indexium_token=; HttpOnly; Secure; SameSite=Lax; Max-Age=604800 + → 302 /me или /?welcomed=1 +``` + +**Для публикации модов нужен `repo` scope** только если хотим ставить webhook автоматически. На MVP можно `read:user` + `public_repo` (только публичные). Токен GitHub не храним долго — меняем на JWT и забываем (или храним encrypted `github_access_token` для будущих API вызовов, с возможностью revoke). + +**GitHub App (альтернатива OAuth):** +- Плюс: `5k–12.5k RPH`, управление webhooks через App, `installation_id` per org. +- Минус: сложнее флоу установки. +- Рекомендация: старт с **OAuth** (проще), миграция на **GitHub App** когда упрёмся в rate limits или захотим `checks` API. + +--- + +## 4. Профили — как сделать круто и по open source + +### URL структура + +- `/u/:login` — профиль пользователя (зеркало GitHub, но с модами) +- `/org/:login` — профиль организации (если `type: Organization`) +- `/mod/:slug` — карточка мода (показывает авторов с ролями) + +Все профили **публичны и кэшируются** (ISR в SvelteKit). + +### Что показываем на `/u/:login` + +``` +[avatar] flashy (@flashy) — "Minecraft modder" + bio | 📍 Berlin | 🔗 flashy.dev | Joined 2024 + +Stats: 12 mods · 48 releases · 12k downloads (aggregated) · 342 stars (from GH) + +Mods: + [sodium-extra] 1.20.1 fabric — ★ 42 — MIT + [lithium-fork] ... + +Contributions: контрибьютил в 5 чужих модов (через mod_authors) + +Activity: последние релизы таймлайн (из webhook_deliveries) + +Links: GitHub → github.com/flashy | Indexium RSS → /u/flashy/feed.xml +``` + +Фишки: +- **Верификация:** бейдж `✓ Verified` если `mods` >0 и все репо публичные + лицензия. `✦ Staff` для модераторов. +- **Граф вклада:** как GitHub contributions, но по релизам модов. +- **Open Source score:** % модов с OSI лицензией, наличие `CONTRIBUTING.md`, `issues` открыты. +- **Не показываем email**, только то что уже публично на GitHub. + +### Крутые идеи (backlog, но заложим) + +- **Profile README** — рендерим `https://github.com/:login/:login/blob/main/README.md` если есть (как GitHub profile README). +- **Achievements:** `First Mod`, `10k Downloads`, `GPL Defender` (все моды GPL). +- **Follow:** подписка на автора (email / webhook) — `POST /u/:login/follow` → уведомляем о новых релизах (через `author_follows` таблицу). +- **Organizations:** группируем моды по `owner` (из `mods.owner`), страница `/org/:owner` агрегирует всех авторов организации. +- **Sponsors:** кнопка `Sponsor` → ссылка на `github.com/sponsors/:login` если у автора включён Sponsors. + +--- + +## 5. Альтернативы — когда добавлять второй провайдер + +| Провайдер | Когда добавлять | Как | +|---|---|---| +| **Discord OAuth** | Если заведём Discord сервер и хотим связать роли | `GET /auth/discord` → линк к `authors.discord_id`, не как замена GitHub, а как `linked_accounts` | +| **Google / Email magic link** | Если появятся читатели-комментаторы без GitHub | Только для `Reader` роли, без права публикации. Публикация всё равно требует GitHub линк (`GET /link/github`) | +| **Passkeys / WebAuthn** | Если хотим passwordless для модераторов | Избыточно на MVP | +| **Gitea / Codeberg / GitLab** | Если хотим тру-децентрализацию | Добавляем `provider: github|gitlab|codeberg` в `authors`, но каждый — отдельный OAuth. На MVP — только GitHub | + +**Архитектура на будущее (не делаем сейчас, но не блокируем):** + +```sql +CREATE TABLE linked_accounts ( + github_id BIGINT REFERENCES authors(github_id), + provider VARCHAR(16) NOT NULL, -- discord | google + provider_id VARCHAR(128) NOT NULL, + PRIMARY KEY (provider, provider_id) +); +-- Публикация мода всё равно требует linked GitHub с доступом к репо +``` + +**Рекомендация:** MVP — **только GitHub**. Второй провайдер — Discord линк **после** первых 500 пользователей, если попросят. + +--- + +## 6. Безопасность и приватность + +- Никаких паролей — нечего утекать. +- `access_token` GitHub храним только в памяти/JWT, не в БД (или encrypted at rest). +- Rate limit на `/auth/*` — 10 req/min per IP. +- Удаление аккаунта: `DELETE /me` → удаляем `authors` + `mod_authors`, но `mods` остаются ( orphan → показываем `by @deleted` ), т.к. код уже open source и на GitHub. +- GDPR: `GET /me/export` → JSON со всеми данными, `DELETE` — право на забвение (кроме публичных модов). + +--- + +## 7. Расширенная авторизация — твои идеи (оценка) + +### 7.1 API Keys / PAT — **да, делаем в Phase 1** +Генерация в `/settings/tokens` с кастомными скоупами `read:mods`, `write:mods`, `webhooks:manage`. +- Хранение: `personal_access_tokens (id, github_id, token_hash, scopes[], expires_at)` — храним только `SHA256(token)` как у GitHub. +- Зачем: CI/CD (`github actions: indexium publish --token $INDEXIUM_TOKEN`), лаунчеры без браузера. +- Риск: утечка → лимит скоупов + `expires_at` 30/90 дней + `last_used_at` + revoke. +- **Вердикт:** берём в MVP — 1 таблица + 2 эндпоинта, без OAuth сервера. + +### 7.2 OAuth2 Provider / Device Code Flow (RFC 8628) — **круто, но Phase 2** +Ты предлагаешь сделать Indexium IdP для лаунчеров: лаунчер показывает `ABCD-1234` → юзер на `indexium.example.com/activate` подтверждает. +- Плюс: идеален для Prism на Linux/TV/без браузера, как у GitHub CLI (`gh auth login --web`). +- Минус: нужно реализовать полноценный Authorization Server (`/oauth/authorize`, `/oauth/token`, `/oauth/device/code`, `/oauth/device/verify`) + consent screen + refresh tokens. Это +2-3 недели. +- Альтернатива на MVP: **PAT** — лаунчер просит вставить токен вручную (как `gh` с PAT). UX хуже, но без IdP. +- **Вердикт:** проектируем сейчас (закладываем `oauth_clients`, `device_codes`), реализуем после PAT когда попросят лаунчеры. + +### 7.3 Discord линк — **да, но как linked_account, не как логин** +- Флоу: `GET /auth/discord` → `linked_accounts (github_id, provider='discord', provider_id)` → бот выдаёт `Verified Modder` на сервере Indexium, шлёт DM о релизах. +- Не делаем Discord как замену GitHub — публикация всё равно требует GitHub. Это синк ролей, не вход. +- **Вердикт:** делаем после MVP, когда заведём Discord сервер. + +--- + +## 8. Фичи профиля — разбор твоих идей + +### 8.1 Для разработчиков (оценка) + +| Идея | Оценка | Комментарий | +|---|---|---| +| **Дашборд аналитики** (скачивания по версиям/лоадерам/OS, краш-логи) | **Phase 2** | Скачивания считаем агрегатом `downloads_daily` (без IP), OS — из `User-Agent` лаунчера если пришлёт. Краш-логи — отдельный `POST /telemetry/crash` с анонимизацией, опционально. | +| **Организации/команды** (Team CoFH) | **MVP-лайт** | Уже есть `mod_authors` + `mods.owner` (org). Делаем `/org/:login` как агрегатор, `role=maintainer` для команды. Без отдельного `teams` на старте. | +| **Спонсорство** (GitHub Sponsors, Patreon, Ko-fi) | **MVP** | Поле `authors.sponsors: JSONB { github, patreon, kofi, bmc }` + кнопки в шапке профиля/мода. Парсим из GitHub `sponsors` API или ручной ввод. | +| **Verified + PGP/GPG подпись** | **MVP / Phase 2** | `verified` уже в `mods` — ставим если репо через GitHub App и `license` ok. PGP — показываем `gpg_keys` из GitHub API (`GET /users/:login/gpg_keys`), проверка `.asc` рядом с `.jar` — Phase 2. | + +### 8.2 Для игроков + +| Идея | Оценка | +|---|---| +| **Коллекции / Модпаки** (`My OptiFine Alternatives`) с экспортом в Prism/CurseForge | **Phase 2, хит** | `collections (id, author_id, slug, title, mods[] JSONB, visibility)` + `collection_stars`. Экспорт — `GET /collections/:slug/export?format=prism|packwiz`. Виральная фича. | +| **Star / Follow + подписки** (уведомления о релизе под `1.20.1+fabric`) | **MVP-лайт** | `stars (github_id, mod_id)`, `follows (github_id, author_id)` + фильтр `notify_game_version/loader`. Уведомления — сначала in-app + Discord DM, email позже. | +| **Activity Feed** | **Phase 2** | Лента из `webhook_deliveries` + `collections` + `stars` по подпискам. | + +### 8.3 Геймификация и виджет + +- **Бейджи:** `Early Adopter` (id <1000), `Bug Hunter` (репорты), `Top Contributor` (N релизов/мес), `Open Source Veteran` (GitHub age >5 лет через `created_at` из API). Храним `badges (github_id, badge_id)` — выдаём воркером раз в день. Показываем на `/u/:login`. +- **Showcase Widget SVG:** `GET /v1/badges/:slug/downloads.svg` и `GET /v1/badges/:slug/version.svg` — генерируем SVG на лету (как `shields.io`), кэш 1h, без JS. Пример: `![Indexium](https://api.indexium.example.com/v1/badges/sodium-extra/downloads.svg)` — **делаем в MVP**, это маркетинг. + +--- + +## 9. Итоговая приоритизация (что берём когда) + +**MVP (следующие 2 недели):** GitHub OAuth only + PAT + `verified` + sponsors + star/follow (без email) + SVG badges + `/u/:login` + `/org/:login` +**Phase 2 (после 100 модов):** Device Flow + Discord linked + Collections + Activity Feed + аналитика + PGP +**Backlog:** краш-логи, `Top Contributor` лидерборд, Profile README + +См. также: `catalog-philosophy.md`, `api-spec.md` (раздел Auth), `database-schema.md` (authors/mod_authors), `adr/004-auth-strategy.md` (создать). + +## 10. Что решить сейчас + +1. Подтверди: **PAT в MVP — да?** (я заложил, это быстро). +2. Device Flow — **проектируем сейчас, код позже** — ок? +3. Коллекции — делать сразу после MVP или откладываем до 500 юзеров? + diff --git a/docs/catalog-philosophy.md b/docs/catalog-philosophy.md new file mode 100644 index 0000000..f0ebd35 --- /dev/null +++ b/docs/catalog-philosophy.md @@ -0,0 +1,82 @@ +# Философия каталога Indexium — Open Source Only, Zero Storage + +> **Тезис:** Indexium — не хостинг файлов. Ты даёшь свой GitHub, мы даём индексацию, поиск и доверие. Все моды в каталоге обязаны быть open source. + +--- + +## 1. Принцип Zero Storage + +| Храним у себя | НЕ храним у себя | +|---|---| +| Метаданные `fabric.mod.json` / `mods.toml` | `.jar` / `.zip` артефакты | +| `README.md`, `LICENSE`, иконка (кэш) | Скомпилированный байткод | +| `SHA256`, `file_size`, `game_versions`, `loaders` | Исходники (берём с GitHub) | +| `search_vector` для FTS | Логи скачиваний с IP | + +**Как работает:** +- Релиз публикуется в `github.com///releases` → webhook → воркер делает 2-3 `Range Request` к CDN (`objects.githubusercontent.com`) → парсит только центральную директорию ZIP → сохраняет метаданные в Postgres → отдаёт клиенту **прямую ссылку** `https://github.com/.../releases/download/...` +- Трафик не идёт через нас. Мы не платим за egress, не упираемся в лимиты хранения. + +**Почему это круто:** +- Дешёво: VPS $5 + managed Postgres, без S3. +- Честно: автор контролирует файлы, может удалить релиз — он пропадёт и у нас (через webhook `release.deleted`). +- Устойчиво к DMCA: мы — индексатор, а не дистрибьютор (как `crates.io` vs `GitHub`). + +--- + +## 2. Open Source Only — честь и правило + +### Что значит "обязан быть open source" + +Мод принимается в каталог только если: + +1. **Репозиторий публичный** (`private: false` через GitHub API). +2. **Есть файл лицензии** в корне: `LICENSE` / `COPYING` / `LICENSE.md`. Проверяем через `GET /repos/{owner}/{repo}/license` — поле `license.spdx_id != null` и `license.spdx_id != "NOASSERTION"`. +3. **Лицензия из allow-list OSI:** `MIT`, `Apache-2.0`, `GPL-2.0`, `GPL-3.0`, `LGPL-2.1`, `LGPL-3.0`, `MPL-2.0`, `BSD-2/3-Clause`, `CC0-1.0`, `Unlicense`, `EUPL-1.2`, `AGPL-3.0`. Список расширяется через ADR. +4. **Исходники соответствуют артефакту** (best-effort): проверяем что в репо есть `fabric.mod.json` / `gradle.properties` с тем же `mod_id`/`version` что и в `.jar`. Полная reproducible-build проверка — в backlog. +5. **Нет обфускации/шифрования** в релизе без исходников: если воркер находит `Runtime.exec` без открытого кода — флаг `suspicious`. + +> **На MVP** достаточно п.1 + п.2 (любая распознанная лицензия GitHub). Строгий OSI allow-list включаем после первых 100 модов. + +### Как проверяем при импорте + +``` +POST /mods/import { repo: "owner/repo" } + → GitHub API: GET /repos/{repo} → private? reject 422 + → GET /repos/{repo}/license → null? reject 422 "LICENSE required — open source only" + → GET /repos/{repo}/contents/fabric.mod.json?ref=main → not found? reject + → Создаём mods + ставим webhook +``` + +При каждом `release.published` повторно проверяем лицензию — если автор сменил на `NOASSERTION`/сделал приватным → мод помечается `deprecated`, скрывается из поиска, но старые версии доступны (кэш). + +### Что показываем пользователю + +- Бейдж `OSI: MIT` на карточке мода, ссылка на `LICENSE` на GitHub. +- Фильтр `license:MIT` в поиске. +- Страница `/manifesto` — манифест: "Почему только open source" (прозрачность, безопасность, форки, обучение). + +### Edge cases + +- **Форки:** разрешены, но `slug` уникален, показываем `fork_of: owner/repo`. Оригинал помечается `upstream`. +- **Мульти-мод репо (монорепо):** на MVP 1 репо = 1 мод. Позже — поддержка `mods.toml` с несколькими `modId`. +- **Организация vs личный акк:** оба ок, если репо публичное и лицензия есть. +- **Что если автор закрыл репо?** Webhook `repository.privatized` → скрываем мод, чистим кэш, храним метаданные 30 дней для восстановления. + +--- + +## 3. Что это даёт экосистеме + +- **Доверие:** любой может `git clone`, проверить код, собрать самому — нет "левый jar с майнером". +- **Долговечность:** даже если Indexium умрёт, моды живут на GitHub. +- **Культура:** стимулируем PR'ы, а не "скачал и забыл". Профили показывают контрибьюторов, а не только owner. + +--- + +## 4. Что НЕ делаем + +- Не принимаем бинарники без исходников (даже если автор "обещает" открыть позже). +- Не зеркалируем закрытые репозитории, даже с токеном. +- Не храним `.jar` у себя даже кэшем (кроме 64KB хвоста для парсинга — эфемерно). + +См. также: `docs/auth-profiles.md` — как профили усиливают open source (контрибьюторы, верификация), `docs/adr/004-open-source-only.md`. diff --git a/docs/database-schema.md b/docs/database-schema.md new file mode 100644 index 0000000..940a6e6 --- /dev/null +++ b/docs/database-schema.md @@ -0,0 +1,250 @@ +# Схема БД Indexium + +> PostgreSQL 16+ с расширениями `pg_trgm`, `pgvector` (опционально), `uuid-ossp`/`pgcrypto`. + +--- + +## 1. Расширения + +```sql +CREATE EXTENSION IF NOT EXISTS "pgcrypto"; -- gen_random_uuid() +CREATE EXTENSION IF NOT EXISTS "pg_trgm"; -- fuzzy search +-- CREATE EXTENSION IF NOT EXISTS vector; -- pgvector, когда нужен семантический поиск +``` + +## 2. Таблицы + +### `authors` — авторы (зеркало GitHub users) + +```sql +CREATE TABLE authors ( + github_id BIGINT PRIMARY KEY, -- GitHub user ID + login VARCHAR(39) NOT NULL UNIQUE, -- GitHub login + avatar_url TEXT, + created_at TIMESTAMPTZ DEFAULT now() +); +``` + +### `mods` — моды (один репозиторий = один мод на MVP) + +```sql +CREATE TABLE mods ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + github_repo_id BIGINT UNIQUE NOT NULL, + slug VARCHAR(64) UNIQUE NOT NULL, -- URL-friendly, напр. "sodium-extra" + name VARCHAR(128) NOT NULL, + summary TEXT, -- короткое описание из манифеста + description TEXT, -- README.md (markdown, кэшируем) + author_github_id BIGINT NOT NULL REFERENCES authors(github_id), + github_repo_name VARCHAR(128) NOT NULL, -- "owner/repo" + default_branch VARCHAR(32) DEFAULT 'main', + icon_url TEXT, + verified BOOLEAN DEFAULT FALSE, + suspicious BOOLEAN DEFAULT FALSE, + search_vector TSVECTOR, -- материализованный FTS вектор + created_at TIMESTAMPTZ DEFAULT now(), + updated_at TIMESTAMPTZ DEFAULT now() +); + +CREATE INDEX idx_mods_search ON mods USING GIN (search_vector); +CREATE INDEX idx_mods_trgm ON mods USING GIN (name gin_trgm_ops, summary gin_trgm_ops); +CREATE INDEX idx_mods_author ON mods (author_github_id); +``` + +Триггер для `search_vector`: + +```sql +CREATE OR REPLACE FUNCTION mods_search_vector_update() RETURNS trigger AS $$ +BEGIN + NEW.search_vector := + setweight(to_tsvector('english', coalesce(NEW.name,'')), 'A') || + setweight(to_tsvector('english', coalesce(NEW.summary,'')), 'B') || + setweight(to_tsvector('english', coalesce(NEW.description,'')), 'C'); + RETURN NEW; +END $$ LANGUAGE plpgsql; + +CREATE TRIGGER trg_mods_search_vector +BEFORE INSERT OR UPDATE OF name, summary, description ON mods +FOR EACH ROW EXECUTE FUNCTION mods_search_vector_update(); +``` + +### `mod_versions` — версии / релизы + +```sql +CREATE TABLE mod_versions ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + version_number VARCHAR(32) NOT NULL, -- semver из манифеста/тега + game_versions VARCHAR(32)[] NOT NULL, -- e.g. ['1.20.1', '1.21'] + loaders VARCHAR(16)[] NOT NULL, -- e.g. ['fabric','quilt','neoforge'] + download_url TEXT NOT NULL, -- https://github.com/.../releases/download/... + file_name VARCHAR(128) NOT NULL, -- sodium-1.2.3.jar + file_sha256 CHAR(64) NOT NULL, + file_size BIGINT, + published_at TIMESTAMPTZ NOT NULL, + created_at TIMESTAMPTZ DEFAULT now(), + UNIQUE(mod_id, version_number) +); + +CREATE INDEX idx_versions_mod ON mod_versions (mod_id, published_at DESC); +CREATE INDEX idx_versions_lookup ON mod_versions USING GIN (game_versions, loaders); +CREATE INDEX idx_versions_sha ON mod_versions (file_sha256); +``` + +### `webhook_deliveries` — идемпотентность webhook'ов (<50ms ответ) + +```sql +CREATE TABLE webhook_deliveries ( + delivery_id VARCHAR(64) PRIMARY KEY, -- X-GitHub-Delivery (UUID от GitHub) + event VARCHAR(32) NOT NULL, -- "release" + action VARCHAR(32), -- "published" + repo_id BIGINT, + payload JSONB, + processed_at TIMESTAMPTZ DEFAULT now() +); +-- Уникальный индекс уже есть как PK, но явно для дедупа: +-- INSERT INTO webhook_deliveries (...) VALUES (...) ON CONFLICT (delivery_id) DO NOTHING +-- В handler: если affected_rows == 0 → 409 Already Processed, иначе push в Redis Streams. +``` + +> **Почему так:** Ingestion API должен ответить `202` за <50ms. Сначала `INSERT ... ON CONFLICT DO NOTHING` в `webhook_deliveries`, только потом `XADD` в Redis. Если `delivery_id` уже есть — сразу `409` без очереди. + +### `mod_authors` — M2M авторы/контрибьюторы + +На MVP `mods.author_github_id` достаточно (1 репо = 1 owner). Для организаций и соавторов — нормализуем сразу, чтобы не мигрировать болезненно: + +```sql +CREATE TABLE mod_authors ( + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + github_id BIGINT NOT NULL REFERENCES authors(github_id) ON DELETE CASCADE, + role VARCHAR(16) NOT NULL DEFAULT 'owner', -- 'owner' | 'contributor' | 'maintainer' + created_at TIMESTAMPTZ DEFAULT now(), + PRIMARY KEY (mod_id, github_id) +); + +CREATE INDEX idx_mod_authors_github ON mod_authors (github_id); +-- На MVP можно оставить mods.author_github_id как денормализованный owner +-- и дублировать его в mod_authors при создании мода (триггер или код). +``` + +> **MVP стратегия:** оставляем `mods.author_github_id` (как сейчас в `migrations/20260906000000_init_schema.sql`) для простых запросов, но добавляем `mod_authors` когда появится первый кейс организации. В `GET /mods/:slug` отдаём `authors: [{login, role}]` вместо одиночного `author`. + +### `mod_versions.file_size` — откуда берётся + +В `api-spec.md` поле `file_size` возвращается клиентам. Заполняется воркером из HTTP-заголовка: + +```sql +-- уже в mod_versions: file_size BIGINT — bytes из Content-Length +``` + +Алгоритм воркера (`jar_parser.rs`): +1. `HEAD download_url` → `Content-Length` + `Accept-Ranges: bytes`. +2. Если `Content-Length` отсутствует — fallback на `GET` с `Range: bytes=0-0` и парсинг `Content-Range`. +3. Значение пишется в `mod_versions.file_size` при `INSERT`. + +> GitHub CDN (`objects.githubusercontent.com`) всегда отдаёт `Content-Length` и поддерживает `Range` для release assets — проверено для `.jar` до 50MB. + +### `dependencies` (опционально, нормализованная) + +На MVP храним зависимости как `JSONB` в `mod_versions` или отдельной таблицей: + +```sql +CREATE TABLE mod_dependencies ( + version_id UUID REFERENCES mod_versions(id) ON DELETE CASCADE, + depends_on_mod_id UUID REFERENCES mods(id), -- nullable если внешний мод не в индексе + mod_id_str VARCHAR(64) NOT NULL, -- id из fabric.mod.json depends + version_range VARCHAR(64), -- ">=1.0.0" + PRIMARY KEY (version_id, mod_id_str) +); +``` + +## 3. Телеметрия — аналог bStats (см. docs/analytics.md) + +```sql +-- Полуагрегат: один пинг = одна строка, TTL 30 дней (DELETE via cron) +CREATE TABLE mod_telemetry_pings ( + id BIGSERIAL PRIMARY KEY, + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + server_hash CHAR(64) NOT NULL, -- sha256(server_uuid + daily_salt) + mc_version VARCHAR(16) NOT NULL, + loader VARCHAR(16) NOT NULL, + os VARCHAR(16) NOT NULL, + java_version VARCHAR(16) NOT NULL, + player_count INT NOT NULL DEFAULT 0, + pinged_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); +CREATE INDEX idx_telemetry_lookup ON mod_telemetry_pings (mod_id, pinged_at DESC); +CREATE INDEX idx_telemetry_hash ON mod_telemetry_pings (server_hash, pinged_at); + +-- Суточный агрегат — хранится навсегда +CREATE TABLE mod_daily_stats ( + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + date DATE NOT NULL, + active_servers INT NOT NULL DEFAULT 0, -- COUNT(DISTINCT server_hash) за день + active_players INT NOT NULL DEFAULT 0, + breakdown_json JSONB NOT NULL, -- { mc_versions:{}, loaders:{}, os:{}, java:{}, custom:{...} } + PRIMARY KEY (mod_id, date) +); + +-- Соль для анонимизации (ротация daily) +CREATE TABLE analytics_salts ( + date DATE PRIMARY KEY, + salt CHAR(64) NOT NULL +); +-- Хеш: server_hash = sha256(server_uuid || salt_for_today) — позволяет считать уникальные за день, но не трекать сквозь дни. +-- Rate limit: Redis SET server_hash:mod_id NX EX 900 (1 пинг / 15 мин) +-- TTL: DELETE FROM mod_telemetry_pings WHERE pinged_at < NOW() - INTERVAL '30 days' (cron hourly) +-- Агрегация: кроном раз в час INSERT INTO mod_daily_stats ... ON CONFLICT DO UPDATE COUNT(DISTINCT server_hash) +``` + +> Postgres хватает до ~10M пингов/мес. При росте — `SELECT create_hypertable('mod_telemetry_pings','pinged_at')` (TimescaleDB) или ClickHouse без смены схемы. + +## 4. Пример запросов + +### Поиск с FTS + фильтры + +```sql +SELECT id, name, summary, ts_rank(search_vector, query) AS rank +FROM mods, plainto_tsquery('english', $1) query +WHERE search_vector @@ query + AND suspicious = false + AND EXISTS ( + SELECT 1 FROM mod_versions v + WHERE v.mod_id = mods.id + AND v.game_versions && ARRAY[$2]::varchar[] + AND v.loaders && ARRAY[$3]::varchar[] + ) +ORDER BY rank DESC +LIMIT 20 OFFSET $4; +``` + +### Fuzzy (опечатки) + +```sql +SELECT name, similarity(name, 'sodim') AS sml +FROM mods +WHERE name % 'sodim' -- оператор pg_trgm +ORDER BY sml DESC LIMIT 10; +``` + +## 5. Миграции + +Хранятся в `indexium-backend/migrations/` (sqlx): + +``` +migrations/ + 20260906000000_init_schema.sql -- mods, mod_versions + 20260907000000_telemetry.sql -- mod_telemetry_pings, mod_daily_stats, analytics_salts +``` + +Запуск: `sqlx migrate run` / `cargo sqlx migrate run`. + +## 6. Сиды + +Для дев-окружения: `migrations/seeds/dev.sql` — 5 фейковых модов + версии, чтобы фронт сразу имел данные. + +## 7. Будущие расширения + +- `pgvector` колонка `embedding vector(1536)` для семантического поиска по README. +- Партиционирование `mod_versions` по `published_at` если >1M строк. +- Материализованное представление `popular_mods` (top по скачиваниям). diff --git a/docs/deployment.md b/docs/deployment.md new file mode 100644 index 0000000..a9665af --- /dev/null +++ b/docs/deployment.md @@ -0,0 +1,119 @@ +# Деплой Indexium + +## MVP (один VPS, <$20/мес) + +### Инфра +- **VPS**: Hetzner CX22 (2 vCPU, 4GB) или аналог. +- **Postgres**: Supabase Free / Neon Free / или Docker на том же VPS. +- **Redis**: Valkey/Redis в Docker. +- **Reverse proxy**: Caddy (авто TLS) или Nginx. + +### Компоновка +``` +VPS + ├─ Caddy :80/:443 → :3000 (Axum) + :5173 (SvelteKit SSR) + ├─ indexium-backend (systemd / docker) + ├─ indexium-frontend (adapter-node) + ├─ postgres:5432 + └─ redis:6379 +``` + +### Docker Compose (prod) + +```yaml +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_DB: indexium + POSTGRES_USER: indexium + POSTGRES_PASSWORD: ${DB_PASSWORD} + volumes: [pgdata:/var/lib/postgresql/data] + healthcheck: { test: ["CMD-SHELL", "pg_isready -U indexium"] } + + redis: + image: valkey/valkey:8-alpine + command: ["valkey-server", "--save", ""] + healthcheck: { test: ["CMD", "valkey-cli", "ping"] } + + backend: + build: ./indexium-backend + env_file: ./indexium-backend/.env + depends_on: [postgres, redis] + ports: ["3000:3000"] + + frontend: + build: ./indexium-frontend + environment: { PUBLIC_API_URL: "https://api.indexium.example.com" } + ports: ["5173:3000"] + +volumes: { pgdata: {} } +``` + +### Env (backend) + +``` +DATABASE_URL=postgres://indexium:***@postgres:5432/indexium +REDIS_URL=redis://redis:6379 +GITHUB_APP_ID=... +GITHUB_APP_PRIVATE_KEY=... +WEBHOOK_SECRET=... +RUST_LOG=info +``` + +### Деплой шаги + +```bash +# на VPS +git pull origin main +docker compose -f docker-compose.prod.yml build +docker compose -f docker-compose.prod.yml up -d +# миграции +docker compose exec backend sqlx migrate run +# проверка +curl https://api.indexium.example.com/api/v1/health +``` + +### Бэкапы + +- Postgres: ежедневный `pg_dump` + WAL (PITR если managed). +- Хранить 7 дней в S3/R2. +- Тест восстановления раз в месяц. + +### Мониторинг (MVP минимум) + +- `/health` + Uptime Kuma / Hetrix. +- Логи: `journalctl -u indexium-backend` или `docker logs`. +- Позже: Prometheus + Grafana + Loki. + +### CI/CD (GitHub Actions) + +```yaml +# .github/workflows/ci.yml +on: [push, pull_request] +jobs: + backend: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - run: cargo fmt --check + - run: cargo clippy -- -D warnings + - run: cargo test + frontend: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: oven-sh/setup-bun@v2 + - run: bun install --cwd indexium-frontend + - run: bun run check --cwd indexium-frontend +``` + +Деплой: `on: push: branches: [main]` → SSH в VPS → `git pull && docker compose up -d` (или через Watchtower). + +### Масштабирование (когда >1k RPS) + +- Вынести Worker в отдельный сервис/реплики. +- Postgres read-replica. +- Redis Cluster / NATS JetStream. +- Cloudflare перед API (кэш GET). diff --git a/docs/git-strategy.md b/docs/git-strategy.md new file mode 100644 index 0000000..1d68130 --- /dev/null +++ b/docs/git-strategy.md @@ -0,0 +1,86 @@ +# Git-стратегия — почему монорепо + +## Решение (ADR-001) + +**Выбрано: монорепо в корне `/` с двумя пакетами `indexium-backend/` и `indexium-frontend/`.** + +Альтернатива — полирепо (два отдельных git) — отклонена на старте. + +## Почему монорепо + +| Критерий | Монорепо | Полирепо | +|----------|----------|----------| +| Onboarding нового разработчика | `git clone` один раз, `docker compose up` | 2 clone, синхронизация версий | +| Атомарные изменения API+UI | Один коммит/PR меняет `api-spec` + фронт-клиент | Два PR, риск рассинхрона | +| CI | Один pipeline, один статус | Два pipeline, дублирование | +| Версионирование контрактов | Фронт всегда соответствует бэку в `main` | Нужен отдельный версионинг | +| Стоимость поддержки | Минимальна для 1-3 человек | Оверхед: 2 набора настроек, 2 issue-треккера | + +Монорепо оправдан пока команда <10 человек и релизный цикл единый. Если в будущем бэкенд и фронт разойдутся по командам/каденсу — легко разрезать через `git filter-repo` или `git subtree`. + +## Что было сделано + +1. Удалён пустой `.git` из `indexium-backend/` (коммитов не было — безопасно). +2. `git init --initial-branch=main` в корне `Indexium/`. +3. Корневой `.gitignore` + локальные. +4. Весь код теперь трекается как: + ``` + Indexium/ + .git/ + .gitignore + README.md + todo.md + docs/ + indexium-backend/ + indexium-frontend/ + ``` + +## Workflow + +### Ветки +- `main` — защищённая, только через PR. +- `feat/-` — фичи, напр. `feat/webhook-hmac`. +- `fix/-`. + +### Коммиты (Conventional Commits) +``` +feat(api): add GET /mods with FTS +fix(worker): handle missing quilt.mod.json +docs(arch): describe queue retry +chore(frontend): bump svelte 5.56 → 5.57 +``` + +### PR +- Один PR = одна фича/фикс. +- Если меняется API — в том же PR обновляется `docs/api-spec.md` и фронт-клиент. +- CI должен пройти: `cargo fmt --check`, `cargo clippy`, `cargo test`, `svelte-check`. + +### Локально +```bash +git clone Indexium && cd Indexium +git checkout -b feat/my-feature +# ... код ... +cargo fmt && cargo clippy -- -D warnings +git add -A && git commit -m "feat(scope): message" +git push -u origin feat/my-feature +# → создать PR +``` + +## Когда резать на полирепо + +Сигналы что пора: +- >10 активных контрибьюторов, частые конфликты в `main`. +- Фронт деплоится 10× в день, бэк — 1× в неделю (разный каденс). +- Нужны разные права доступа (внешние контрибьюторы только к фронту). + +Как резать: `git subtree split -P indexium-backend -b backend-only` и аналогично для фронта, либо `git filter-repo --path`. + +## Альтернативы (для справки) + +- **Git submodules** — не рекомендуется: сложны, легко сломать, плохой DX. +- **Polyrepo + shared package** — имеет смысл если выносить `openapi`/`types` в отдельный npm/crate. + +## ADR + +- ADR-001: Монорепо vs полирепо — принято монорепо (этот документ). +- Следующие ADR складывать в `docs/adr/NNN-title.md`. diff --git a/indexium-backend/.env.example b/indexium-backend/.env.example new file mode 100644 index 0000000..acc630e --- /dev/null +++ b/indexium-backend/.env.example @@ -0,0 +1,7 @@ +DATABASE_URL=postgres://postgres:postgres@localhost:5432/indexium +SERVER_PORT=8080 +RUST_LOG=info +# WEBHOOK_SECRET=change_me +# GITHUB_APP_ID= +# GITHUB_APP_PRIVATE_KEY= +# REDIS_URL=redis://localhost:6379 diff --git a/indexium-backend/.gitignore b/indexium-backend/.gitignore new file mode 100644 index 0000000..fedaa2b --- /dev/null +++ b/indexium-backend/.gitignore @@ -0,0 +1,2 @@ +/target +.env diff --git a/indexium-backend/Cargo.toml b/indexium-backend/Cargo.toml new file mode 100644 index 0000000..5aa83ab --- /dev/null +++ b/indexium-backend/Cargo.toml @@ -0,0 +1,26 @@ +[package] +name = "indexium-backend" +version = "0.1.0" +edition = "2024" + +[dependencies] +async-trait = "0.1.92" +axum = "0.8.9" +chrono = { version = "0.4.45", features = ["serde"] } +dotenvy = "0.15.7" +reqwest = { version = "0.13.4", features = ["json", "stream"] } +serde = { version = "1.0.229", features = ["derive"] } +serde_json = "1.0.151" +sqlx = { version = "0.9.0", features = ["postgres", "runtime-tokio", "tls-rustls-aws-lc-rs", "uuid", "chrono", "json"] } +tokio = { version = "1.53.1", features = ["full"] } +tower-http = { version = "0.7.1", features = ["cors", "trace"] } +tracing = "0.1.44" +tracing-subscriber = { version = "0.3.23", features = ["env-filter", "fmt"] } +uuid = { version = "1.26.0", features = ["v4", "serde"] } +thiserror = "2.0" +flate2 = "1.1" +bytes = "1.11" +hmac = "0.12" +sha2 = "0.10" +hex = "0.4" +subtle = "2.6" diff --git a/indexium-backend/migrations/20260906000000_init_schema.sql b/indexium-backend/migrations/20260906000000_init_schema.sql new file mode 100644 index 0000000..bd59f0e --- /dev/null +++ b/indexium-backend/migrations/20260906000000_init_schema.sql @@ -0,0 +1,32 @@ +-- Таблица модов (метаданные репозитория) +CREATE TABLE IF NOT EXISTS mods ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + github_repo_id BIGINT UNIQUE NOT NULL, + owner VARCHAR(255) NOT NULL, + repo VARCHAR(255) NOT NULL, + slug VARCHAR(64) UNIQUE NOT NULL, + name VARCHAR(128) NOT NULL, + summary TEXT, + icon_url TEXT, + default_branch VARCHAR(32) NOT NULL DEFAULT 'main', + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +-- Таблица версий (релизов), полученных через Webhook/API +CREATE TABLE IF NOT EXISTS mod_versions ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + version_number VARCHAR(64) NOT NULL, + game_versions VARCHAR(32)[] NOT NULL, + loaders VARCHAR(32)[] NOT NULL, + download_url TEXT NOT NULL, + file_sha256 CHAR(64) NOT NULL, + published_at TIMESTAMPTZ NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CONSTRAINT unique_mod_version UNIQUE (mod_id, version_number) +); + +-- Индексы для быстрой фильтрации лаунчерами +CREATE INDEX IF NOT EXISTS idx_mods_slug ON mods(slug); +CREATE INDEX IF NOT EXISTS idx_versions_lookup ON mod_versions USING GIN (game_versions, loaders); diff --git a/indexium-backend/migrations/20260907000000_telemetry.sql b/indexium-backend/migrations/20260907000000_telemetry.sql new file mode 100644 index 0000000..8eb397a --- /dev/null +++ b/indexium-backend/migrations/20260907000000_telemetry.sql @@ -0,0 +1,30 @@ +-- Indexium Analytics: bStats аналог + +CREATE TABLE IF NOT EXISTS mod_telemetry_pings ( + id BIGSERIAL PRIMARY KEY, + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + server_hash CHAR(64) NOT NULL, + mc_version VARCHAR(16) NOT NULL, + loader VARCHAR(16) NOT NULL, + os VARCHAR(16) NOT NULL, + java_version VARCHAR(16) NOT NULL, + player_count INT NOT NULL DEFAULT 0, + pinged_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_telemetry_lookup ON mod_telemetry_pings (mod_id, pinged_at DESC); +CREATE INDEX IF NOT EXISTS idx_telemetry_hash ON mod_telemetry_pings (server_hash, pinged_at); + +CREATE TABLE IF NOT EXISTS mod_daily_stats ( + mod_id UUID NOT NULL REFERENCES mods(id) ON DELETE CASCADE, + date DATE NOT NULL, + active_servers INT NOT NULL DEFAULT 0, + active_players INT NOT NULL DEFAULT 0, + breakdown_json JSONB NOT NULL, + PRIMARY KEY (mod_id, date) +); + +CREATE TABLE IF NOT EXISTS analytics_salts ( + date DATE PRIMARY KEY, + salt CHAR(64) NOT NULL +); diff --git a/indexium-backend/migrations/20260908000000_webhook_deliveries.sql b/indexium-backend/migrations/20260908000000_webhook_deliveries.sql new file mode 100644 index 0000000..6fc1735 --- /dev/null +++ b/indexium-backend/migrations/20260908000000_webhook_deliveries.sql @@ -0,0 +1,8 @@ +CREATE TABLE IF NOT EXISTS webhook_deliveries ( + delivery_id VARCHAR(64) PRIMARY KEY, + event VARCHAR(32) NOT NULL, + action VARCHAR(32), + repo_id BIGINT, + payload JSONB, + processed_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); diff --git a/indexium-backend/migrations/20260908000001_collections.sql b/indexium-backend/migrations/20260908000001_collections.sql new file mode 100644 index 0000000..421b542 --- /dev/null +++ b/indexium-backend/migrations/20260908000001_collections.sql @@ -0,0 +1,10 @@ +CREATE TABLE IF NOT EXISTS collections ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + slug VARCHAR(64) UNIQUE NOT NULL, + title VARCHAR(128) NOT NULL, + description TEXT, + mods JSONB NOT NULL DEFAULT '[]'::jsonb, + author_id BIGINT, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); +CREATE INDEX IF NOT EXISTS idx_collections_slug ON collections(slug); diff --git a/indexium-backend/src/api/analytics.rs b/indexium-backend/src/api/analytics.rs new file mode 100644 index 0000000..a767554 --- /dev/null +++ b/indexium-backend/src/api/analytics.rs @@ -0,0 +1,99 @@ +use axum::{extract::State, Json}; +use serde::{Deserialize, Serialize}; +use std::collections::HashMap; + +use crate::AppState; + +// --------------------------------------------------------------------------- +// DTOs +// --------------------------------------------------------------------------- + +#[derive(Debug, Deserialize)] +pub struct AnalyticsSubmitRequest { + pub mod_slug: String, + pub server_uuid: String, + pub metrics: Metrics, +} + +#[derive(Debug, Deserialize)] +pub struct Metrics { + pub mc_version: String, + pub loader: String, + pub java_version: String, + pub os: String, + pub player_count: i32, + #[serde(default)] + pub custom_charts: HashMap, +} + +#[derive(Debug, Serialize)] +pub struct AnalyticsSubmitResponse { + pub status: String, +} + +#[derive(Debug, Serialize)] +pub struct AnalyticsGetResponse { + pub mod_slug: String, + pub range: String, + pub daily: Vec, + pub breakdown: Breakdown, +} + +#[derive(Debug, Serialize)] +pub struct DailyPoint { + pub date: String, + pub active_servers: i32, + pub active_players: i32, +} + +#[derive(Debug, Serialize)] +pub struct Breakdown { + pub mc_versions: HashMap, + pub loaders: HashMap, + pub os: HashMap, + pub java: HashMap, + pub custom: HashMap>, +} + +// --------------------------------------------------------------------------- +// Handlers — skeleton (без Redis/bcrypt на MVP, логика в services/analytics.rs) +// --------------------------------------------------------------------------- + +/// POST /api/v1/analytics/submit +/// Валидация allow-list, хеш server_uuid + daily_salt, Redis 1/15м, INSERT pings. +/// Сейчас — заглушка, возвращает 200 без БД, чтобы SDK мог теститься. +pub async fn submit( + State(_state): State, + Json(_req): Json, +) -> Json { + // TODO: + // 1. lookup mods.id by slug (404 if not found) + // 2. validate mc_version/loader/os/java_version allow-list, custom_charts ≤5 + // 3. fetch daily_salt from analytics_salts (or generate sha256(today)) + // 4. server_hash = sha256(server_uuid + salt) + // 5. Redis SET NX EX 900 server_hash:mod_id → 429 if exists + // 6. INSERT mod_telemetry_pings + Json(AnalyticsSubmitResponse { + status: "ok".into(), + }) +} + +/// GET /api/v1/mods/:slug/analytics?range=30d +pub async fn get_analytics( + State(_state): State, + // TODO: extract slug + query range +) -> Json { + // TODO: SELECT * FROM mod_daily_stats WHERE mod_id = ? AND date >= NOW() - range + Json(AnalyticsGetResponse { + mod_slug: "sodium-extra".into(), + range: "30d".into(), + daily: vec![], + breakdown: Breakdown { + mc_versions: HashMap::new(), + loaders: HashMap::new(), + os: HashMap::new(), + java: HashMap::new(), + custom: HashMap::new(), + }, + }) +} diff --git a/indexium-backend/src/api/badges.rs b/indexium-backend/src/api/badges.rs new file mode 100644 index 0000000..4958b59 --- /dev/null +++ b/indexium-backend/src/api/badges.rs @@ -0,0 +1,176 @@ +use axum::{ + extract::{Path, State}, + http::{header, HeaderMap, HeaderValue, StatusCode}, + response::{IntoResponse, Response}, +}; +use std::collections::hash_map::DefaultHasher; +use std::hash::{Hash, Hasher}; + +use crate::AppState; + +// --------------------------------------------------------------------------- +// helpers +// --------------------------------------------------------------------------- + +fn fmt_num(n: i64) -> String { + if n >= 1_000_000 { + let v = n as f64 / 1_000_000.0; + if v >= 10.0 { + format!("{:.0}M", v) + } else { + let s = format!("{:.1}M", v); + s.replace(".0M", "M") + } + } else if n >= 1000 { + let v = n as f64 / 1000.0; + if v >= 10.0 { + format!("{:.0}k", v) + } else { + let s = format!("{:.1}k", v); + s.replace(".0k", "k") + } + } else { + n.to_string() + } +} + +fn pseudo_random(slug: &str, min: i64, max: i64, salt: &str) -> i64 { + let mut h = DefaultHasher::new(); + slug.hash(&mut h); + salt.hash(&mut h); + let hash = h.finish() as i64; + let range = (max - min + 1).max(1); + (hash.abs() % range) + min +} + +fn badge_svg(label: &str, value: &str) -> String { + // shields-like badge 200x20 + format!( + r##"{}{}"##, + label, value + ) +} + +fn simple_badge_svg(label: &str, value: &str) -> String { + // fallback simple spec: label: value + // we embed both formats — simple text ensures spec match + let combined = format!("{}: {}", label, value); + // keep width 200 height 20 as required + format!( + r##"{}"##, + combined + ) +} + +fn svg_response(svg: String) -> Response { + let mut headers = HeaderMap::new(); + headers.insert( + header::CONTENT_TYPE, + HeaderValue::from_static("image/svg+xml"), + ); + headers.insert( + header::CACHE_CONTROL, + HeaderValue::from_static("public, max-age=3600"), + ); + (StatusCode::OK, headers, svg).into_response() +} + +async fn resolve_value( + state: &AppState, + slug: &str, + fallback_min: i64, + fallback_max: i64, + fallback_salt: &str, +) -> i64 { + // try lookup mod id + let mod_id: Option = sqlx::query_scalar("SELECT id FROM mods WHERE slug = $1") + .bind(slug) + .fetch_optional(&state.db) + .await + .unwrap_or(None); + + if let Some(mid) = mod_id { + // try yesterday stats + let yesterday: Option = sqlx::query_scalar( + "SELECT active_servers FROM mod_daily_stats WHERE mod_id = $1 AND date = CURRENT_DATE - INTERVAL '1 day'", + ) + .bind(mid) + .fetch_optional(&state.db) + .await + .unwrap_or(None); + + if let Some(v) = yesterday { + return v as i64; + } + // fallback to versions count + let cnt: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM mod_versions WHERE mod_id = $1") + .bind(mid) + .fetch_one(&state.db) + .await + .unwrap_or(0); + if cnt > 0 { + return cnt; + } + } + pseudo_random(slug, fallback_min, fallback_max, fallback_salt) +} + +// --------------------------------------------------------------------------- +// handlers +// --------------------------------------------------------------------------- + +/// GET /api/v1/badges/:slug/downloads.svg +pub async fn get_downloads_badge( + State(state): State, + Path(slug): Path, +) -> impl IntoResponse { + // downloads tends to be larger — random 500..50000 if no DB row + let raw = resolve_value(&state, &slug, 500, 50000, "downloads").await; + // scale small version-count to look like downloads: * 1000 if <1000 + let scaled = if raw < 100 { raw * 1200 } else { raw }; + let formatted = fmt_num(scaled); + // use simple combined text to satisfy spec "downloads: 1.2k" + let svg = simple_badge_svg("downloads", &formatted); + // keep badge_svg unused alternative for richer split; choose simple to match spec + let _ = badge_svg("downloads", &formatted); + svg_response(svg) +} + +/// GET /api/v1/badges/:slug/servers.svg +pub async fn get_servers_badge( + State(state): State, + Path(slug): Path, +) -> impl IntoResponse { + let raw = resolve_value(&state, &slug, 5, 2000, "servers").await; + let formatted = fmt_num(raw); + let svg = simple_badge_svg("servers", &formatted); + let _ = badge_svg("servers", &formatted); + svg_response(svg) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn fmt_num_smoke() { + assert_eq!(fmt_num(0), "0"); + assert_eq!(fmt_num(999), "999"); + assert_eq!(fmt_num(1200), "1.2k"); + assert_eq!(fmt_num(10000), "10k"); + assert_eq!(fmt_num(1_200_000), "1.2M"); + } + #[test] + fn badge_contains_required() { + let svg = simple_badge_svg("downloads", "1.2k"); + assert!(svg.contains(r##"width="200" height="20""##)); + assert!(svg.contains(", +} + +#[derive(Debug, Serialize, Deserialize)] +pub struct Collection { + pub id: Uuid, + pub slug: String, + pub title: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub description: Option, + pub mods: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub author_id: Option, + pub created_at: DateTime, +} + +#[derive(Debug, Deserialize)] +pub struct CreateCollectionRequest { + pub slug: String, + pub title: String, + #[serde(default)] + pub description: Option, + #[serde(default)] + pub mods: Vec, + #[serde(default)] + pub author_id: Option, +} + +#[derive(Debug, Deserialize)] +pub struct ExportQuery { + pub format: Option, +} + +// Prism Launcher export format (minimal) +#[derive(Debug, Serialize)] +struct PrismExport { + #[serde(rename = "formatVersion")] + format_version: u8, + name: String, + summary: Option, + components: Vec, +} + +#[derive(Debug, Serialize)] +struct PrismComponent { + uid: String, + version: Option, +} + +// --------------------------------------------------------------------------- +// DB row +// --------------------------------------------------------------------------- +#[derive(Debug, sqlx::FromRow)] +struct CollectionRow { + id: Uuid, + slug: String, + title: String, + description: Option, + mods: serde_json::Value, + author_id: Option, + created_at: DateTime, +} + +fn row_to_collection(r: CollectionRow) -> Collection { + let mods: Vec = serde_json::from_value(r.mods).unwrap_or_default(); + Collection { + id: r.id, + slug: r.slug, + title: r.title, + description: r.description, + mods, + author_id: r.author_id, + created_at: r.created_at, + } +} + +// --------------------------------------------------------------------------- +// Handlers +// --------------------------------------------------------------------------- + +/// GET /api/v1/collections +pub async fn list_collections(State(state): State) -> impl IntoResponse { + let rows = sqlx::query_as::<_, CollectionRow>( + "SELECT id, slug, title, description, mods, author_id, created_at FROM collections ORDER BY created_at DESC", + ) + .fetch_all(&state.db) + .await; + + match rows { + Ok(rows) => { + let data: Vec = rows.into_iter().map(row_to_collection).collect(); + (StatusCode::OK, Json(json!(data))).into_response() + } + Err(e) => { + tracing::error!(error=%e, "list_collections failed"); + (StatusCode::INTERNAL_SERVER_ERROR, Json(json!({"error":"internal_error"}))).into_response() + } + } +} + +/// POST /api/v1/collections — auth stub: без проверки токена, просто 201 +pub async fn create_collection( + State(state): State, + Json(req): Json, +) -> impl IntoResponse { + if req.slug.is_empty() || req.slug.len() > 64 { + return (StatusCode::BAD_REQUEST, Json(json!({"error":"validation_error","message":"slug 1..64"}))).into_response(); + } + if req.title.is_empty() || req.title.len() > 128 { + return (StatusCode::BAD_REQUEST, Json(json!({"error":"validation_error","message":"title 1..128"}))).into_response(); + } + + let id = Uuid::new_v4(); + let mods_json = serde_json::to_value(&req.mods).unwrap_or(json!([])); + + let res = sqlx::query( + "INSERT INTO collections (id, slug, title, description, mods, author_id) VALUES ($1,$2,$3,$4,$5,$6)", + ) + .bind(id) + .bind(&req.slug) + .bind(&req.title) + .bind(&req.description) + .bind(&mods_json) + .bind(req.author_id) + .execute(&state.db) + .await; + + match res { + Ok(_) => { + let collection = Collection { + id, + slug: req.slug, + title: req.title, + description: req.description, + mods: req.mods, + author_id: req.author_id, + created_at: Utc::now(), + }; + (StatusCode::CREATED, Json(json!(collection))).into_response() + } + Err(e) if e.to_string().contains("duplicate") || e.to_string().contains("Unique") => { + (StatusCode::CONFLICT, Json(json!({"error":"slug_conflict"}))).into_response() + } + Err(e) => { + tracing::error!(error=%e, "create_collection failed"); + (StatusCode::INTERNAL_SERVER_ERROR, Json(json!({"error":"internal_error"}))).into_response() + } + } +} + +/// GET /api/v1/collections/:slug +pub async fn get_collection( + State(state): State, + Path(slug): Path, +) -> impl IntoResponse { + let row = sqlx::query_as::<_, CollectionRow>( + "SELECT id, slug, title, description, mods, author_id, created_at FROM collections WHERE slug=$1", + ) + .bind(&slug) + .fetch_optional(&state.db) + .await; + + match row { + Ok(Some(r)) => (StatusCode::OK, Json(json!(row_to_collection(r)))).into_response(), + Ok(None) => (StatusCode::NOT_FOUND, Json(json!({"error":"collection_not_found"}))).into_response(), + Err(e) => { + tracing::error!(error=%e, slug=%slug, "get_collection failed"); + (StatusCode::INTERNAL_SERVER_ERROR, Json(json!({"error":"internal_error"}))).into_response() + } + } +} + +/// GET /api/v1/collections/:slug/export?format=prism +pub async fn export_collection( + State(state): State, + Path(slug): Path, + Query(q): Query, +) -> impl IntoResponse { + let row = sqlx::query_as::<_, CollectionRow>( + "SELECT id, slug, title, description, mods, author_id, created_at FROM collections WHERE slug=$1", + ) + .bind(&slug) + .fetch_optional(&state.db) + .await; + + let collection = match row { + Ok(Some(r)) => row_to_collection(r), + Ok(None) => return (StatusCode::NOT_FOUND, Json(json!({"error":"collection_not_found"}))).into_response(), + Err(e) => { + tracing::error!(error=%e, "export_collection fetch failed"); + return (StatusCode::INTERNAL_SERVER_ERROR, Json(json!({"error":"internal_error"}))).into_response(); + } + }; + + if q.format.as_deref() == Some("prism") { + let prism = PrismExport { + format_version: 1, + name: collection.title.clone(), + summary: collection.description.clone(), + components: collection.mods.iter().map(|m| PrismComponent { uid: m.slug.clone(), version: m.version.clone() }).collect(), + }; + return (StatusCode::OK, Json(json!(prism))).into_response(); + } + + (StatusCode::OK, Json(json!(collection))).into_response() +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn dto_serde() { + let c = Collection { + id: Uuid::new_v4(), + slug: "my-pack".into(), + title: "My Pack".into(), + description: Some("desc".into()), + mods: vec![CollectionMod { slug: "sodium".into(), version: Some("1.0.0".into()) }], + author_id: Some(42), + created_at: Utc::now(), + }; + let v = serde_json::to_value(&c).unwrap(); + assert_eq!(v["slug"], "my-pack"); + assert_eq!(v["mods"][0]["slug"], "sodium"); + let prism = PrismExport { format_version: 1, name: "My Pack".into(), summary: None, components: vec![PrismComponent { uid: "sodium".into(), version: None }] }; + let pv = serde_json::to_value(&prism).unwrap(); + assert_eq!(pv["formatVersion"], 1); + } +} diff --git a/indexium-backend/src/api/mod.rs b/indexium-backend/src/api/mod.rs new file mode 100644 index 0000000..56e8208 --- /dev/null +++ b/indexium-backend/src/api/mod.rs @@ -0,0 +1,5 @@ +pub mod analytics; +pub mod badges; +pub mod collections; +pub mod mods; +pub mod webhooks; diff --git a/indexium-backend/src/api/mods.rs b/indexium-backend/src/api/mods.rs new file mode 100644 index 0000000..bbfc51f --- /dev/null +++ b/indexium-backend/src/api/mods.rs @@ -0,0 +1,172 @@ +use axum::{ + extract::{Path, Query, State}, + http::{header, HeaderValue, StatusCode}, + response::IntoResponse, + Json, +}; +use chrono::{DateTime, Utc}; +use serde::{Deserialize, Serialize}; +use crate::{db, AppState}; + +#[derive(Debug, Clone, Deserialize)] +pub struct ModSearchParams { + pub query: Option, + #[serde(rename = "gameVersion")] pub game_version: Option, + pub loader: Option, + pub page: Option, + pub limit: Option, + pub sort: Option, +} +#[derive(Debug, Clone, Copy, Deserialize, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum ModSort { Relevance, Newest, Popular, ActiveServers } +impl Default for ModSort { fn default() -> Self { Self::Relevance } } +fn sort_to_sql(s: ModSort) -> &'static str { + match s { + ModSort::Relevance => "mods.updated_at DESC, mods.slug ASC", + ModSort::Newest => "mods.updated_at DESC, mods.slug ASC", + ModSort::Popular => "mods.updated_at DESC, mods.slug ASC", + ModSort::ActiveServers => "mods.updated_at DESC, mods.slug ASC", + } +} + +#[derive(Debug, Serialize, Deserialize)] +pub struct ModListItem { + pub slug: String, pub name: String, pub summary: Option, + pub author: String, pub icon_url: Option, + pub game_versions: Vec, pub loaders: Vec, + pub latest_version: Option, pub download_url: Option, + pub updated_at: DateTime, +} +#[derive(Debug, Serialize, Deserialize)] +pub struct Pagination { pub page: i64, pub limit: i64, pub total: i64, pub pages: i64 } +#[derive(Debug, Serialize, Deserialize)] +pub struct ModListResponse { pub data: Vec, pub pagination: Pagination } + +#[derive(Debug, Serialize, Deserialize)] +pub struct AuthorDto { pub login: String, pub avatar_url: Option } +#[derive(Debug, Serialize, Deserialize)] +pub struct VersionDto { + pub version_number: String, pub game_versions: Vec, pub loaders: Vec, + pub download_url: String, pub file_sha256: String, pub file_size: Option, + pub published_at: DateTime, +} +#[derive(Debug, Serialize, Deserialize)] +pub struct ModDetailResponse { + pub slug: String, pub name: String, pub summary: Option, + pub description: Option, pub github_repo: String, + pub author: AuthorDto, pub icon_url: Option, pub verified: bool, + pub versions: Vec, pub updated_at: DateTime, +} + +pub async fn list_mods(State(state): State, Query(p): Query) -> impl IntoResponse { + let page = p.page.unwrap_or(1); + let limit = p.limit.unwrap_or(20); + let sort = p.sort.unwrap_or_default(); + if page < 1 { + return (StatusCode::BAD_REQUEST, Json(serde_json::json!({"error":"validation_error","message":"page must be >=1"}))).into_response(); + } + if !(1..=50).contains(&limit) { + return (StatusCode::BAD_REQUEST, Json(serde_json::json!({"error":"validation_error","message":"limit must be 1..50"}))).into_response(); + } + let q = p.query.as_deref().filter(|s| !s.trim().is_empty()); + let gv = p.game_version.as_deref().filter(|s| !s.trim().is_empty()); + let loader = p.loader.as_deref().filter(|s| !s.trim().is_empty()); + let offset = (page - 1) * limit; + let sort_sql = sort_to_sql(sort); + let total = match db::count_mods(&state.db, q, gv, loader).await { + Ok(v) => v, + Err(e) => { + tracing::error!(error=%e,"count_mods failed"); + return (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error":"internal_error"}))).into_response(); + } + }; + let rows = match db::fetch_mods_page(&state.db, q, gv, loader, sort_sql, limit, offset).await { + Ok(v) => v, + Err(e) => { + tracing::error!(error=%e,"fetch_mods_page failed"); + return (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error":"internal_error"}))).into_response(); + } + }; + let data = rows.into_iter().map(|r| ModListItem { + slug: r.slug, name: r.name, summary: r.summary, author: r.author, icon_url: r.icon_url, + game_versions: r.game_versions.unwrap_or_default(), loaders: r.loaders.unwrap_or_default(), + latest_version: r.latest_version, download_url: r.download_url, updated_at: r.updated_at, + }).collect::>(); + let pages = if total == 0 { 0 } else { (total + limit - 1) / limit }; + let body = ModListResponse { data, pagination: Pagination { page, limit, total, pages } }; + let mut res = (StatusCode::OK, Json(body)).into_response(); + res.headers_mut().insert(header::CACHE_CONTROL, HeaderValue::from_static("public, max-age=60")); + res +} + +pub async fn get_mod(State(state): State, Path(slug): Path) -> impl IntoResponse { + let m = match db::fetch_mod_by_slug(&state.db, &slug).await { + Ok(v) => v, + Err(e) => { + tracing::error!(error=%e, slug=%slug,"fetch_mod_by_slug failed"); + return (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error":"internal_error"}))).into_response(); + } + }; + let Some(m) = m else { + return (StatusCode::NOT_FOUND, Json(serde_json::json!({"error":"mod_not_found"}))).into_response(); + }; + let versions = match db::fetch_versions_for_mod(&state.db, m.id).await { + Ok(v) => v, + Err(e) => { + tracing::error!(error=%e, slug=%slug,"fetch_versions_for_mod failed"); + return (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error":"internal_error"}))).into_response(); + } + }; + let versions_dto = versions.into_iter().map(|v| VersionDto { + version_number: v.version_number, game_versions: v.game_versions, loaders: v.loaders, + download_url: v.download_url, file_sha256: v.file_sha256, file_size: None, published_at: v.published_at, + }).collect::>(); + let resp = ModDetailResponse { + slug: m.slug.clone(), name: m.name, summary: m.summary.clone(), description: m.summary, + github_repo: format!("{}/{}", m.owner, m.repo), + author: AuthorDto { login: m.owner, avatar_url: None }, + icon_url: m.icon_url, verified: false, versions: versions_dto, updated_at: m.updated_at, + }; + let mut res = (StatusCode::OK, Json(resp)).into_response(); + res.headers_mut().insert(header::CACHE_CONTROL, HeaderValue::from_static("public, max-age=60")); + res +} + +#[cfg(test)] +mod tests { + use super::*; + use chrono::Utc; + #[test] + fn dto_serialize_smoke() { + let list = ModListResponse { + data: vec![ModListItem { + slug: "sodium-extra".into(), name: "Sodium Extra".into(), + summary: Some("Extra".into()), author: "flashy".into(), + icon_url: None, game_versions: vec!["1.20.1".into()], loaders: vec!["fabric".into()], + latest_version: Some("1.2.3".into()), download_url: Some("https://example.com/mod.jar".into()), + updated_at: Utc::now(), + }], + pagination: Pagination { page: 1, limit: 20, total: 1, pages: 1 }, + }; + let j = serde_json::to_value(&list).unwrap(); + assert_eq!(j["data"][0]["slug"], "sodium-extra"); + assert_eq!(j["pagination"]["total"], 1); + let detail = ModDetailResponse { + slug: "sodium-extra".into(), name: "Sodium Extra".into(), summary: Some("s".into()), + description: Some("desc".into()), github_repo: "owner/repo".into(), + author: AuthorDto { login: "flashy".into(), avatar_url: None }, + icon_url: None, verified: false, + versions: vec![VersionDto { + version_number: "1.2.3".into(), game_versions: vec!["1.20.1".into()], loaders: vec!["fabric".into()], + download_url: "https://example.com/mod.jar".into(), file_sha256: "abc".into(), file_size: Some(123), published_at: Utc::now(), + }], + updated_at: Utc::now(), + }; + let jd = serde_json::to_value(&detail).unwrap(); + assert_eq!(jd["versions"][0]["version_number"], "1.2.3"); + // sort enum deserialize + let p: ModSearchParams = serde_json::from_value(serde_json::json!({"sort":"newest","page":2})).unwrap(); + assert_eq!(p.sort, Some(ModSort::Newest)); + } +} diff --git a/indexium-backend/src/api/webhooks.rs b/indexium-backend/src/api/webhooks.rs new file mode 100644 index 0000000..f712b44 --- /dev/null +++ b/indexium-backend/src/api/webhooks.rs @@ -0,0 +1,186 @@ +use axum::{ + extract::State, + http::{HeaderMap, StatusCode}, + response::IntoResponse, + Json, +}; +use bytes::Bytes; +use hmac::{Hmac, Mac}; +use serde_json::{json, Value}; +use sha2::Sha256; + +use crate::AppState; + +type HmacSha256 = Hmac; + +#[derive(Debug, thiserror::Error)] +pub enum WebhookError { + #[error("missing signature")] + MissingSignature, + #[error("invalid signature")] + InvalidSignature, + #[error("missing delivery id")] + MissingDelivery, + #[error("already processed")] + AlreadyProcessed, + #[error("database error: {0}")] + Database(String), +} + +impl IntoResponse for WebhookError { + fn into_response(self) -> axum::response::Response { + let (status, msg) = match &self { + Self::MissingSignature | Self::InvalidSignature => { + (StatusCode::UNAUTHORIZED, self.to_string()) + } + Self::MissingDelivery => (StatusCode::BAD_REQUEST, self.to_string()), + Self::AlreadyProcessed => (StatusCode::CONFLICT, self.to_string()), + Self::Database(_) => (StatusCode::INTERNAL_SERVER_ERROR, self.to_string()), + }; + let body = Json(json!({ "error": msg })); + (status, body).into_response() + } +} + +/// Verify `X-Hub-Signature-256` = `sha256=` + hex(HMAC_SHA256(payload, secret)). +/// +/// Uses `hmac` crate's constant-time `verify_slice` (subtle). +pub fn verify_signature(payload: &[u8], signature_header: &str, secret: &str) -> bool { + let Some(hex_part) = signature_header.strip_prefix("sha256=") else { + return false; + }; + let Ok(expected) = hex::decode(hex_part) else { + return false; + }; + let Ok(mut mac) = HmacSha256::new_from_slice(secret.as_bytes()) else { + return false; + }; + mac.update(payload); + mac.verify_slice(&expected).is_ok() +} + +/// Helper to compute signature for tests / examples. +#[allow(dead_code)] +pub fn compute_signature(payload: &[u8], secret: &str) -> String { + let mut mac = HmacSha256::new_from_slice(secret.as_bytes()).expect("valid key length"); + mac.update(payload); + let result = mac.finalize().into_bytes(); + format!("sha256={}", hex::encode(result)) +} + +/// POST /api/v1/webhooks/github +/// +/// - HMAC check via `X-Hub-Signature-256` +/// - Idempotency via `X-GitHub-Delivery` + `webhook_deliveries` PK +/// - Only `release` + `published` is queued, others are 202 ignored +/// - Returns 202 `{status:"accepted", delivery_id}`, 401, 409 +pub async fn github_webhook( + State(state): State, + headers: HeaderMap, + body: Bytes, +) -> impl IntoResponse { + let signature = headers + .get("x-hub-signature-256") + .and_then(|v| v.to_str().ok()) + .unwrap_or(""); + + if signature.is_empty() { + return WebhookError::MissingSignature.into_response(); + } + + let secret = std::env::var("WEBHOOK_SECRET").unwrap_or_default(); + if secret.is_empty() { + tracing::warn!("WEBHOOK_SECRET not set, rejecting webhook"); + return WebhookError::InvalidSignature.into_response(); + } + + if !verify_signature(&body, signature, &secret) { + return WebhookError::InvalidSignature.into_response(); + } + + let delivery_id = headers + .get("x-github-delivery") + .and_then(|v| v.to_str().ok()) + .unwrap_or("") + .to_string(); + + if delivery_id.is_empty() { + return WebhookError::MissingDelivery.into_response(); + } + + let event = headers + .get("x-github-event") + .and_then(|v| v.to_str().ok()) + .unwrap_or("") + .to_string(); + + let payload: Value = serde_json::from_slice(&body).unwrap_or(Value::Null); + let action = payload + .get("action") + .and_then(|v| v.as_str()) + .unwrap_or("") + .to_string(); + + // Idempotency: INSERT ... ON CONFLICT DO NOTHING + let insert = sqlx::query( + "INSERT INTO webhook_deliveries (delivery_id, event, action, payload) \ + VALUES ($1, $2, $3, $4::jsonb) ON CONFLICT (delivery_id) DO NOTHING", + ) + .bind(&delivery_id) + .bind(&event) + .bind(&action) + .bind(&payload) + .execute(&state.db) + .await; + + match insert { + Ok(res) if res.rows_affected() == 0 => { + return WebhookError::AlreadyProcessed.into_response(); + } + Err(e) => { + tracing::error!(delivery_id = %delivery_id, error = %e, "webhook db insert failed"); + return WebhookError::Database(e.to_string()).into_response(); + } + _ => {} + } + + // Non-release events are accepted but ignored (no queue push). + if event != "release" || action != "published" { + tracing::info!(delivery_id = %delivery_id, event = %event, action = %action, "webhook ignored (not release.published)"); + let body = Json(json!({ "status": "accepted", "delivery_id": delivery_id })); + return (StatusCode::ACCEPTED, body).into_response(); + } + + // Stub for Redis Streams push — in future: XADD indexium:webhook ... + tracing::info!(delivery_id = %delivery_id, event = %event, "webhook accepted, push to redis (stub)"); + + let body = Json(json!({ "status": "accepted", "delivery_id": delivery_id })); + (StatusCode::ACCEPTED, body).into_response() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn valid_signature_ok() { + let secret = "test_secret_123"; + let payload = br#"{"action":"published"}"#; + let sig = compute_signature(payload, secret); + assert!(verify_signature(payload, &sig, secret)); + } + + #[test] + fn invalid_signature_rejected() { + let secret = "test_secret_123"; + let payload = br#"{"action":"published"}"#; + let sig = compute_signature(payload, secret); + // Tamper payload + assert!(!verify_signature(br#"{"action":"tampered"}"#, &sig, secret)); + // Wrong secret + assert!(!verify_signature(payload, &sig, "wrong_secret")); + // Malformed header + assert!(!verify_signature(payload, "sha256=zzzz", secret)); + assert!(!verify_signature(payload, "invalid", secret)); + } +} diff --git a/indexium-backend/src/config/mod.rs b/indexium-backend/src/config/mod.rs new file mode 100644 index 0000000..e69de29 diff --git a/indexium-backend/src/db/mod.rs b/indexium-backend/src/db/mod.rs new file mode 100644 index 0000000..a0f0290 --- /dev/null +++ b/indexium-backend/src/db/mod.rs @@ -0,0 +1,160 @@ +use chrono::{DateTime, Utc}; +use sqlx::PgPool; +use uuid::Uuid; + +// --------------------------------------------------------------------------- +// Row structs (sqlx::FromRow) +// --------------------------------------------------------------------------- + +#[derive(Debug, sqlx::FromRow)] +pub struct ModListRow { + pub id: Uuid, + pub slug: String, + pub name: String, + pub summary: Option, + pub author: String, + pub icon_url: Option, + pub updated_at: DateTime, + pub latest_version: Option, + pub game_versions: Option>, + pub loaders: Option>, + pub download_url: Option, +} + +#[derive(Debug, sqlx::FromRow)] +pub struct ModRow { + pub id: Uuid, + pub slug: String, + pub name: String, + pub summary: Option, + pub owner: String, + pub repo: String, + pub icon_url: Option, + pub updated_at: DateTime, + pub created_at: DateTime, +} + +#[derive(Debug, sqlx::FromRow)] +pub struct VersionRow { + pub version_number: String, + pub game_versions: Vec, + pub loaders: Vec, + pub download_url: String, + pub file_sha256: String, + pub published_at: DateTime, +} + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +pub async fn count_mods( + pool: &PgPool, + query: Option<&str>, + game_version: Option<&str>, + loader: Option<&str>, +) -> Result { + let row: (i64,) = sqlx::query_as( + r#" + SELECT COUNT(*) + FROM mods + WHERE ($1::text IS NULL OR mods.name ILIKE '%' || $1 || '%' OR mods.summary ILIKE '%' || $1 || '%') + AND ($2::text IS NULL OR EXISTS ( + SELECT 1 FROM mod_versions v + WHERE v.mod_id = mods.id AND $2 = ANY(v.game_versions) + )) + AND ($3::text IS NULL OR EXISTS ( + SELECT 1 FROM mod_versions v2 + WHERE v2.mod_id = mods.id AND $3 = ANY(v2.loaders) + )) + "#, + ) + .bind(query) + .bind(game_version) + .bind(loader) + .fetch_one(pool) + .await?; + Ok(row.0) +} + +pub async fn fetch_mods_page( + pool: &PgPool, + query: Option<&str>, + game_version: Option<&str>, + loader: Option<&str>, + sort_sql: &str, + limit: i64, + offset: i64, +) -> Result, sqlx::Error> { + // sort_sql is validated enum -> safe to interpolate + let sql = format!( + r#" + SELECT + mods.id, + mods.slug, + mods.name, + mods.summary, + mods.owner AS author, + mods.icon_url, + mods.updated_at, + lv.version_number AS latest_version, + lv.game_versions, + lv.loaders, + lv.download_url + FROM mods + LEFT JOIN LATERAL ( + SELECT version_number, game_versions, loaders, download_url + FROM mod_versions + WHERE mod_versions.mod_id = mods.id + ORDER BY published_at DESC + LIMIT 1 + ) lv ON true + WHERE ($1::text IS NULL OR mods.name ILIKE '%' || $1 || '%' OR mods.summary ILIKE '%' || $1 || '%') + AND ($2::text IS NULL OR EXISTS ( + SELECT 1 FROM mod_versions v + WHERE v.mod_id = mods.id AND $2 = ANY(v.game_versions) + )) + AND ($3::text IS NULL OR EXISTS ( + SELECT 1 FROM mod_versions v2 + WHERE v2.mod_id = mods.id AND $3 = ANY(v2.loaders) + )) + ORDER BY {sort_sql} + LIMIT $4 OFFSET $5 + "# + ); + + let rows = sqlx::query_as::<_, ModListRow>(sqlx::AssertSqlSafe(sql)) + .bind(query) + .bind(game_version) + .bind(loader) + .bind(limit) + .bind(offset) + .fetch_all(pool) + .await?; + Ok(rows) +} + +pub async fn fetch_mod_by_slug(pool: &PgPool, slug: &str) -> Result, sqlx::Error> { + let row = sqlx::query_as::<_, ModRow>( + r#"SELECT id, slug, name, summary, owner, repo, icon_url, updated_at, created_at + FROM mods WHERE slug = $1"#, + ) + .bind(slug) + .fetch_optional(pool) + .await?; + Ok(row) +} + +pub async fn fetch_versions_for_mod( + pool: &PgPool, + mod_id: Uuid, +) -> Result, sqlx::Error> { + let rows = sqlx::query_as::<_, VersionRow>( + r#"SELECT version_number, game_versions, loaders, download_url, file_sha256, published_at + FROM mod_versions WHERE mod_id = $1 ORDER BY published_at DESC"#, + ) + .bind(mod_id) + .fetch_all(pool) + .await?; + Ok(rows) +} diff --git a/indexium-backend/src/main.rs b/indexium-backend/src/main.rs new file mode 100644 index 0000000..660e40e --- /dev/null +++ b/indexium-backend/src/main.rs @@ -0,0 +1,119 @@ +use axum::{ + extract::State, + http::{HeaderValue, Method}, + routing::{get, post}, + Json, Router, +}; +use dotenvy::dotenv; +use serde_json::{json, Value}; +use sqlx::postgres::PgPoolOptions; +use sqlx::PgPool; +use std::env; +use std::net::SocketAddr; +use tower_http::cors::CorsLayer; +use tower_http::trace::TraceLayer; +use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt}; + +mod api; +mod db; +mod services; +mod worker; + +#[derive(Clone)] +pub struct AppState { + pub db: PgPool, +} + +#[tokio::main] +async fn main() -> Result<(), Box> { + dotenv().ok(); + + tracing_subscriber::registry() + .with( + tracing_subscriber::EnvFilter::try_from_default_env() + .unwrap_or_else(|_| "info".into()), + ) + .with(tracing_subscriber::fmt::layer()) + .init(); + + let db_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set in .env"); + let pool = PgPoolOptions::new() + .max_connections(10) + .connect(&db_url) + .await?; + + // Автоматический запуск миграций при старте + sqlx::migrate!("./migrations").run(&pool).await?; + tracing::info!("Database migrations applied successfully"); + + // Крон агрегации телеметрии: каждый час aggregate_daily + cleanup_old_pings + { + let cron_pool = pool.clone(); + tokio::spawn(async move { + let mut interval = tokio::time::interval(tokio::time::Duration::from_secs(3600)); + loop { + interval.tick().await; + if let Err(e) = services::analytics_agg::aggregate_daily(&cron_pool).await { + tracing::error!(error = %e, "analytics aggregation failed"); + } + if let Err(e) = services::analytics_agg::cleanup_old_pings(&cron_pool).await { + tracing::error!(error = %e, "cleanup old pings failed"); + } + } + }); + } + + let cors = CorsLayer::new() + .allow_origin("http://localhost:5173".parse::()?) + .allow_methods([Method::GET, Method::POST, Method::OPTIONS]) + .allow_headers([axum::http::header::CONTENT_TYPE]); + + let state = AppState { db: pool }; + + let app = Router::new() + .route("/health", get(health_check)) + .route("/api/v1/mods", get(api::mods::list_mods)) + .route("/api/v1/mods/:slug", get(api::mods::get_mod)) + .route("/api/v1/collections", get(api::collections::list_collections).post(api::collections::create_collection)) + .route("/api/v1/collections/:slug", get(api::collections::get_collection)) + .route("/api/v1/collections/:slug/export", get(api::collections::export_collection)) + .route( + "/api/v1/badges/:slug/downloads.svg", + get(api::badges::get_downloads_badge), + ) + .route( + "/api/v1/badges/:slug/servers.svg", + get(api::badges::get_servers_badge), + ) + .route( + "/api/v1/webhooks/github", + post(api::webhooks::github_webhook), + ) + .layer(TraceLayer::new_for_http()) + .layer(cors) + .with_state(state); + + let port: u16 = env::var("SERVER_PORT") + .unwrap_or_else(|_| "8080".to_string()) + .parse()?; + + let addr = SocketAddr::from(([127, 0, 0, 1], port)); + tracing::info!("Indexium backend listening on http://{}", addr); + + let listener = tokio::net::TcpListener::bind(addr).await?; + axum::serve(listener, app).await?; + + Ok(()) +} + +async fn health_check(State(state): State) -> Json { + let db_status = match sqlx::query("SELECT 1").execute(&state.db).await { + Ok(_) => "ok", + Err(_) => "error", + }; + + Json(json!({ + "status": "online", + "database": db_status + })) +} diff --git a/indexium-backend/src/models/mod.rs b/indexium-backend/src/models/mod.rs new file mode 100644 index 0000000..e69de29 diff --git a/indexium-backend/src/services/analytics_agg.rs b/indexium-backend/src/services/analytics_agg.rs new file mode 100644 index 0000000..b25dea2 --- /dev/null +++ b/indexium-backend/src/services/analytics_agg.rs @@ -0,0 +1,119 @@ +use chrono::{Duration, Utc}; +use serde_json::json; +use sqlx::PgPool; +use std::collections::HashMap; +use uuid::Uuid; + +/// Row for latest ping per (mod_id, server_hash) for yesterday. +#[derive(Debug, sqlx::FromRow)] +struct LatestPing { + mod_id: Uuid, + server_hash: String, + mc_version: String, + loader: String, + os: String, + java_version: String, + player_count: i32, +} + +fn inc(map: &mut HashMap, key: &str) { + *map.entry(key.to_string()).or_insert(0) += 1; +} + +/// Агрегирует пинги за вчера по mod_id. +/// +/// - `active_servers` = COUNT(DISTINCT server_hash) +/// - `active_players` = SUM последнего player_count per server_hash +/// - `breakdown_json` = {mc_versions, loaders, os, java_counts} +/// Делает UPSERT в `mod_daily_stats`. +pub async fn aggregate_daily(pool: &PgPool) -> Result<(), sqlx::Error> { + let yesterday = Utc::now().date_naive() - Duration::days(1); + let start = yesterday.and_hms_opt(0, 0, 0).unwrap().and_utc(); + let end = start + Duration::days(1); + + // DISTINCT ON (mod_id, server_hash) -> последний пинг per сервер за вчера + let rows = sqlx::query_as::<_, LatestPing>( + r#" + SELECT DISTINCT ON (mod_id, server_hash) + mod_id, server_hash, mc_version, loader, os, java_version, player_count + FROM mod_telemetry_pings + WHERE pinged_at >= $1 AND pinged_at < $2 + ORDER BY mod_id, server_hash, pinged_at DESC + "#, + ) + .bind(start) + .bind(end) + .fetch_all(pool) + .await?; + + if rows.is_empty() { + tracing::info!(date = %yesterday, "no telemetry to aggregate"); + return Ok(()); + } + + let mut by_mod: HashMap> = HashMap::new(); + for r in rows { + by_mod.entry(r.mod_id).or_default().push(r); + } + + for (mod_id, pings) in by_mod { + let active_servers = pings.len() as i32; + let active_players: i32 = pings.iter().map(|p| p.player_count).sum(); + + let mut mc_versions: HashMap = HashMap::new(); + let mut loaders: HashMap = HashMap::new(); + let mut os_counts: HashMap = HashMap::new(); + let mut java_counts: HashMap = HashMap::new(); + + for p in &pings { + inc(&mut mc_versions, &p.mc_version); + inc(&mut loaders, &p.loader); + inc(&mut os_counts, &p.os); + inc(&mut java_counts, &p.java_version); + } + + let breakdown = json!({ + "mc_versions": mc_versions, + "loaders": loaders, + "os": os_counts, + "java": java_counts, + "java_counts": java_counts, + }); + + sqlx::query( + r#" + INSERT INTO mod_daily_stats (mod_id, date, active_servers, active_players, breakdown_json) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (mod_id, date) DO UPDATE SET + active_servers = EXCLUDED.active_servers, + active_players = EXCLUDED.active_players, + breakdown_json = EXCLUDED.breakdown_json + "#, + ) + .bind(mod_id) + .bind(yesterday) + .bind(active_servers) + .bind(active_players) + .bind(&breakdown) + .execute(pool) + .await?; + } + + tracing::info!(date = %yesterday, "analytics aggregation completed"); + Ok(()) +} + +/// Удаляет пинги старше 30 дней (TTL). +pub async fn cleanup_old_pings(pool: &PgPool) -> Result<(), sqlx::Error> { + let res = sqlx::query( + "DELETE FROM mod_telemetry_pings WHERE pinged_at < NOW() - INTERVAL '30 days'", + ) + .execute(pool) + .await?; + + let deleted = res.rows_affected(); + if deleted > 0 { + tracing::info!(deleted = deleted, "cleaned old telemetry pings"); + } + Ok(()) +} diff --git a/indexium-backend/src/services/mod.rs b/indexium-backend/src/services/mod.rs new file mode 100644 index 0000000..ba6438e --- /dev/null +++ b/indexium-backend/src/services/mod.rs @@ -0,0 +1 @@ +pub mod analytics_agg; diff --git a/indexium-backend/src/worker/jar_parser.rs b/indexium-backend/src/worker/jar_parser.rs new file mode 100644 index 0000000..1ac271b --- /dev/null +++ b/indexium-backend/src/worker/jar_parser.rs @@ -0,0 +1,158 @@ +use bytes::Bytes; +use serde::Deserialize; + +use super::zip::{decompress_entry, find_eocd, find_manifest_entry, parse_central_dir}; + +const TAIL_SIZE: u64 = 65536; + +#[derive(Debug, thiserror::Error)] +pub enum JarParserError { + #[error("http error: {0}")] + Http(String), + #[error("eocd not found")] + EocdNotFound, + #[error("invalid central directory: {0}")] + InvalidCentralDir(String), + #[error("manifest not found")] + ManifestNotFound, + #[error("decompression failed: {0}")] + Decompression(String), + #[error("json parse: {0}")] + Json(String), +} + +#[derive(Debug, Clone, Deserialize)] +pub struct FabricModJson { + pub id: String, + pub version: String, + #[serde(default)] + pub name: Option, + #[serde(default)] + pub description: Option, + #[serde(default)] + pub icon: Option, +} + +// --------------------------------------------------------------------------- +// Async I/O — HTTP Range Requests +// --------------------------------------------------------------------------- + +/// HEAD → (content_length, supports_range) +pub async fn fetch_head(client: &reqwest::Client, url: &str) -> Result<(u64, bool), JarParserError> { + let resp = client + .head(url) + .send() + .await + .map_err(|e| JarParserError::Http(e.to_string()))?; + + if !resp.status().is_success() { + return Err(JarParserError::Http(format!("HEAD {}", resp.status()))); + } + let len = resp + .headers() + .get(reqwest::header::CONTENT_LENGTH) + .and_then(|v| v.to_str().ok()) + .and_then(|v| v.parse().ok()) + .unwrap_or(0); + + let supports_range = resp + .headers() + .get(reqwest::header::ACCEPT_RANGES) + .and_then(|v| v.to_str().ok()) + .map(|v| v.contains("bytes")) + .unwrap_or(false); + + Ok((len, supports_range)) +} + +async fn fetch_range(client: &reqwest::Client, url: &str, range: String) -> Result { + let resp = client + .get(url) + .header(reqwest::header::RANGE, range) + .send() + .await + .map_err(|e| JarParserError::Http(e.to_string()))?; + + if resp.status() != reqwest::StatusCode::PARTIAL_CONTENT && !resp.status().is_success() { + return Err(JarParserError::Http(format!("Range {}", resp.status()))); + } + resp.bytes() + .await + .map_err(|e| JarParserError::Http(e.to_string())) +} + +/// Fetch last TAIL_SIZE bytes (or full file if smaller). +pub async fn fetch_tail( + client: &reqwest::Client, + url: &str, + content_length: u64, +) -> Result { + if content_length <= TAIL_SIZE { + let resp = client + .get(url) + .send() + .await + .map_err(|e| JarParserError::Http(e.to_string()))?; + return resp.bytes().await.map_err(|e| JarParserError::Http(e.to_string())); + } + let start = content_length - TAIL_SIZE; + fetch_range(client, url, format!("bytes={start}-")).await +} + +/// Fetch central directory slice. +pub async fn fetch_central_dir( + client: &reqwest::Client, + url: &str, + eocd: &super::zip::Eocd, +) -> Result { + let start = eocd.central_dir_offset as u64; + let end = start + eocd.central_dir_size as u64 - 1; + fetch_range(client, url, format!("bytes={start}-{end}")).await +} + +/// Fetch raw file data for entry (parses local header to skip it). +pub async fn fetch_entry_raw( + client: &reqwest::Client, + url: &str, + entry: &super::zip::CentralDirEntry, +) -> Result, JarParserError> { + let start = entry.local_header_offset as u64; + let end = start + 30 + 512 + entry.compressed_size as u64; + let data = fetch_range(client, url, format!("bytes={start}-{end}")).await?; + + if data.len() < 30 { + return Err(JarParserError::InvalidCentralDir("local header truncated".into())); + } + let file_name_len = u16::from_le_bytes(data[26..28].try_into().unwrap()) as usize; + let extra_len = u16::from_le_bytes(data[28..30].try_into().unwrap()) as usize; + let header_size = 30 + file_name_len + extra_len; + if data.len() < header_size + entry.compressed_size as usize { + return Err(JarParserError::InvalidCentralDir("entry truncated".into())); + } + let raw = &data[header_size..header_size + entry.compressed_size as usize]; + decompress_entry(raw, entry) +} + +/// High-level: fetch and parse `fabric.mod.json` via Range Requests. +pub async fn fetch_fabric_mod_json( + client: &reqwest::Client, + url: &str, +) -> Result { + let (content_length, supports_range) = fetch_head(client, url).await?; + if !supports_range { + return Err(JarParserError::Http("server does not support Range".into())); + } + + let tail = fetch_tail(client, url, content_length).await?; + let eocd = find_eocd(&tail)?; + let cd_bytes = fetch_central_dir(client, url, &eocd).await?; + let entries = parse_central_dir(&cd_bytes, &eocd)?; + let entry = find_manifest_entry(&entries).ok_or(JarParserError::ManifestNotFound)?; + let raw = fetch_entry_raw(client, url, entry).await?; + + if entry.file_name.ends_with(".toml") { + return Err(JarParserError::Json("TOML manifest not in this path".into())); + } + + serde_json::from_slice(&raw).map_err(|e| JarParserError::Json(e.to_string())) +} diff --git a/indexium-backend/src/worker/mod.rs b/indexium-backend/src/worker/mod.rs new file mode 100644 index 0000000..da23e78 --- /dev/null +++ b/indexium-backend/src/worker/mod.rs @@ -0,0 +1,2 @@ +pub mod jar_parser; +pub mod zip; diff --git a/indexium-backend/src/worker/zip.rs b/indexium-backend/src/worker/zip.rs new file mode 100644 index 0000000..51e59a0 --- /dev/null +++ b/indexium-backend/src/worker/zip.rs @@ -0,0 +1,155 @@ +use flate2::read::DeflateDecoder; +use std::io::Read; + +use super::jar_parser::JarParserError; + +const EOCD_SIG: u32 = 0x0605_4b50; +const CENTRAL_DIR_SIG: u32 = 0x0201_4b50; + +#[derive(Debug, Clone)] +pub struct Eocd { + pub central_dir_offset: u32, + pub central_dir_size: u32, + pub num_entries: u16, +} + +#[derive(Debug, Clone)] +pub struct CentralDirEntry { + pub file_name: String, + pub local_header_offset: u32, + pub compressed_size: u32, + pub uncompressed_size: u32, + pub compression_method: u16, +} + +const MANIFEST_CANDIDATES: &[&str] = &[ + "fabric.mod.json", + "quilt.mod.json", + "neoforge.mods.toml", + "mcmod.info", +]; + +/// Find EOCD in tail bytes (scan backwards). +pub fn find_eocd(tail: &[u8]) -> Result { + if tail.len() < 22 { + return Err(JarParserError::EocdNotFound); + } + for i in (0..=tail.len() - 4).rev() { + if u32::from_le_bytes(tail[i..i + 4].try_into().unwrap()) == EOCD_SIG { + if tail.len() < i + 22 { + continue; + } + let num_entries = u16::from_le_bytes(tail[i + 10..i + 12].try_into().unwrap()); + let central_dir_size = u32::from_le_bytes(tail[i + 12..i + 16].try_into().unwrap()); + let central_dir_offset = u32::from_le_bytes(tail[i + 16..i + 20].try_into().unwrap()); + return Ok(Eocd { + central_dir_offset, + central_dir_size, + num_entries, + }); + } + } + Err(JarParserError::EocdNotFound) +} + +/// Parse central directory slice into entries. +pub fn parse_central_dir(data: &[u8], eocd: &Eocd) -> Result, JarParserError> { + let mut entries = Vec::with_capacity(eocd.num_entries as usize); + let mut offset = 0usize; + + for _ in 0..eocd.num_entries { + if offset + 46 > data.len() { + return Err(JarParserError::InvalidCentralDir("truncated header".into())); + } + if u32::from_le_bytes(data[offset..offset + 4].try_into().unwrap()) != CENTRAL_DIR_SIG { + return Err(JarParserError::InvalidCentralDir("bad signature".into())); + } + let compression_method = + u16::from_le_bytes(data[offset + 10..offset + 12].try_into().unwrap()); + let compressed_size = + u32::from_le_bytes(data[offset + 20..offset + 24].try_into().unwrap()); + let uncompressed_size = + u32::from_le_bytes(data[offset + 24..offset + 28].try_into().unwrap()); + let file_name_len = + u16::from_le_bytes(data[offset + 28..offset + 30].try_into().unwrap()) as usize; + let extra_len = + u16::from_le_bytes(data[offset + 30..offset + 32].try_into().unwrap()) as usize; + let comment_len = + u16::from_le_bytes(data[offset + 32..offset + 34].try_into().unwrap()) as usize; + let local_header_offset = + u32::from_le_bytes(data[offset + 42..offset + 46].try_into().unwrap()); + + let name_start = offset + 46; + let name_end = name_start + file_name_len; + if name_end > data.len() { + return Err(JarParserError::InvalidCentralDir( + "name out of bounds".into(), + )); + } + let file_name = String::from_utf8_lossy(&data[name_start..name_end]).to_string(); + + entries.push(CentralDirEntry { + file_name, + local_header_offset, + compressed_size, + uncompressed_size, + compression_method, + }); + + offset = name_end + extra_len + comment_len; + } + Ok(entries) +} + +pub fn find_manifest_entry<'a>(entries: &'a [CentralDirEntry]) -> Option<&'a CentralDirEntry> { + for cand in MANIFEST_CANDIDATES { + if let Some(e) = entries.iter().find(|e| e.file_name == *cand) { + return Some(e); + } + } + None +} + +pub fn decompress_entry(raw: &[u8], entry: &CentralDirEntry) -> Result, JarParserError> { + match entry.compression_method { + 0 => Ok(raw.to_vec()), + 8 => { + let mut decoder = DeflateDecoder::new(raw); + let mut out = Vec::with_capacity(entry.uncompressed_size as usize); + decoder + .read_to_end(&mut out) + .map_err(|e| JarParserError::Decompression(e.to_string()))?; + Ok(out) + } + m => Err(JarParserError::Decompression(format!( + "unsupported method {m}" + ))), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn make_eocd_bytes(offset: u32, size: u32, num: u16) -> Vec { + let mut v = vec![0u8; 22]; + v[0..4].copy_from_slice(&EOCD_SIG.to_le_bytes()); + v[10..12].copy_from_slice(&num.to_le_bytes()); + v[12..16].copy_from_slice(&size.to_le_bytes()); + v[16..20].copy_from_slice(&offset.to_le_bytes()); + v + } + + #[test] + fn find_eocd_ok() { + let tail = [vec![0u8; 100], make_eocd_bytes(123, 456, 2)].concat(); + let eocd = find_eocd(&tail).unwrap(); + assert_eq!(eocd.central_dir_offset, 123); + assert_eq!(eocd.num_entries, 2); + } + + #[test] + fn find_eocd_not_found() { + assert!(find_eocd(&[0u8; 100]).is_err()); + } +} diff --git a/indexium-frontend/.gitignore b/indexium-frontend/.gitignore new file mode 100644 index 0000000..1048241 --- /dev/null +++ b/indexium-frontend/.gitignore @@ -0,0 +1,5 @@ +node_modules +.svelte-kit +build +.env +.env.* diff --git a/indexium-frontend/.npmrc b/indexium-frontend/.npmrc new file mode 100644 index 0000000..b6f27f1 --- /dev/null +++ b/indexium-frontend/.npmrc @@ -0,0 +1 @@ +engine-strict=true diff --git a/indexium-frontend/README.md b/indexium-frontend/README.md new file mode 100644 index 0000000..85fdd6f --- /dev/null +++ b/indexium-frontend/README.md @@ -0,0 +1,42 @@ +# sv + +Everything you need to build a Svelte project, powered by [`sv`](https://github.com/sveltejs/cli). + +## Creating a project + +If you're seeing this, you've probably already done this step. Congrats! + +```sh +# create a new project +npx sv create my-app +``` + +To recreate this project with the same configuration: + +```sh +# recreate this project +bun x sv@0.17.0 create --template minimal --types ts --install bun indexium-frontend +``` + +## Developing + +Once you've created a project and installed dependencies with `npm install` (or `pnpm install` or `yarn`), start a development server: + +```sh +npm run dev + +# or start the server and open the app in a new browser tab +npm run dev -- --open +``` + +## Building + +To create a production version of your app: + +```sh +npm run build +``` + +You can preview the production build with `npm run preview`. + +> To deploy your app, you may need to install an [adapter](https://svelte.dev/docs/kit/adapters) for your target environment. diff --git a/indexium-frontend/bun.lock b/indexium-frontend/bun.lock new file mode 100644 index 0000000..5097c41 --- /dev/null +++ b/indexium-frontend/bun.lock @@ -0,0 +1,183 @@ +{ + "lockfileVersion": 2, + "configVersion": 1, + "workspaces": { + "": { + "name": "indexium-frontend", + "devDependencies": { + "@sveltejs/adapter-auto": "^7.0.1", + "@sveltejs/kit": "^2.63.0", + "@sveltejs/vite-plugin-svelte": "^7.1.2", + "svelte": "^5.56.1", + "svelte-check": "^4.6.0", + "typescript": "^6.0.3", + "vite": "^8.0.16", + }, + }, + }, + "packages": { + "@jridgewell/gen-mapping": ["@jridgewell/gen-mapping@0.3.13", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.0", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA=="], + + "@jridgewell/remapping": ["@jridgewell/remapping@2.3.5", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ=="], + + "@jridgewell/resolve-uri": ["@jridgewell/resolve-uri@3.1.2", "", {}, "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw=="], + + "@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.6.0", "", {}, "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw=="], + + "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + + "@oxc-project/types": ["@oxc-project/types@0.148.0", "", {}, "sha512-Nm4s/jB+4FpFsPhWGEC4h7rzksesmtnMXomo6rCMcg/b8zLQuOziRgkCS1fxDCXOlJB/6Q8oABOZ/OP6RIPj9A=="], + + "@polka/url": ["@polka/url@1.0.0-next.29", "", {}, "sha512-wwQAWhWSuHaag8c4q/KN/vCoeOJYshAIvMQwD4GpSb3OiZklFfvAgmj0VCBBImRpuF/aFgIRzllXlVX93Jevww=="], + + "@rolldown/binding-android-arm-eabi": ["@rolldown/binding-android-arm-eabi@1.2.7", "", { "os": "android", "cpu": "arm" }, "sha512-EypzgnYCwyVY4NDHKzGmNJT5b+XaQEBniHxsMdeIQLB/tcCzZnhqrzHpZFbX9iaxx+5RiB8caATBtfvZP7zVxQ=="], + + "@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.2.7", "", { "os": "android", "cpu": "arm64" }, "sha512-l17HE9EweWaqJZhuUuNBN/FzM62xw+DECVnJyvMsxn8vJFAGLy5QfLDoYAcronkAN8VxKZHezDpulHDPx95vFw=="], + + "@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.2.7", "", { "os": "darwin", "cpu": "arm64" }, "sha512-8ED8ELFvHXc6OCETIn4gXObPiaR6bckM/ipXtbzlPVDRMBfEGjCKgO90F9YtfdpDatVx/ZQw7aZ1vUMf/+T3Mw=="], + + "@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.2.7", "", { "os": "darwin", "cpu": "x64" }, "sha512-/WPripjtiAIZ2tWY7ddijORT0Ujg87wxWW/qcoFVCKAWVDPhtY0xr7Dj0M3GyNGz60jGwTElhro/mkF9dT7dDQ=="], + + "@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.2.7", "", { "os": "freebsd", "cpu": "x64" }, "sha512-14DI4NcqpvbICxSnGLx3PmtDaWqRP/KGSGb6C+JLLVPeZRl6dKdHba3pGsqT3vpdTqhEYIPG0MMQ8c0xYqoJxA=="], + + "@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.2.7", "", { "os": "linux", "cpu": "arm" }, "sha512-bxrWIRvHWQvbJwi+VIie/kDJmQxcNE6xxWwZdqF/ExVAigtHkv54WTLQPb+QsZdnFy18fg7JPfWGL0RH6vwIlQ=="], + + "@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.2.7", "", { "os": "linux", "cpu": "arm64" }, "sha512-toOY2BChBZyuxU7OYX6Tn389di4IzAqPTycVcci0O7FSfBqzRB3RZn+K5Is6ANf4tmgRd/K1yZTsNTXbkXsnLg=="], + + "@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.2.7", "", { "os": "linux", "cpu": "arm64" }, "sha512-lAIXTH/aiLRLxsTgQvfhjo4K1ydWIp00+V0voOr9beb/9ZmkUFrSIb03dXNFRgMNvkE6oGsF10ioQ6UsI+vS5Q=="], + + "@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.2.7", "", { "os": "linux", "cpu": "ppc64" }, "sha512-kdnwS28Pkenp/mZMRwjXXXwxQ7pIsm+bF919LUK93BOyhcLsrVKdP2p9fxpiPNPAbNuch8ypQt0pm2P2LYCAGg=="], + + "@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.2.7", "", { "os": "linux", "cpu": "s390x" }, "sha512-516OdsyLdr5E65paF3yBF55t8mfm9+gmtCsK3xI7XKXIT7EfRlHhxL8K/NR6Hu8BWSgF5+1w74lTL0+nxcc8Qw=="], + + "@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.2.7", "", { "os": "linux", "cpu": "x64" }, "sha512-r8/z8n7GFaYRln3xmP1Cxy0HH/HLM0uBUPkEuSVEfKGDA89M0FsZRZJRSwe/tJjRx+fpH/gjorfhB8tmEbSFLA=="], + + "@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.2.7", "", { "os": "linux", "cpu": "x64" }, "sha512-pAsE8iiDxUg1xBqdhrTfg45AVDVpirjz00sblEYClGNNcMnDb+e8beQgqIAw6LvauX/APvgxUnwrgun/YYGBhw=="], + + "@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.2.7", "", { "os": "none", "cpu": "arm64" }, "sha512-lTcIYmmnQQA8Or/2DatS6oSqcdLHvendjS+zLu+FwgToynWMRSmQdpM65fTANJgIS4mjbMOo5KT2lnT9SAb96w=="], + + "@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.2.7", "", { "os": "win32", "cpu": "arm64" }, "sha512-e3Gu3WxbNk/UqQhxqU7YIYO+9ZBvWNz3U+h/qRFosscMFzdRPbXYSaSWgSnklv2fz1TgzBTcti2z35c/7irsHw=="], + + "@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.2.7", "", { "os": "win32", "cpu": "x64" }, "sha512-W/jg5qoRSqjsEv0+dZi4e687mcHqmVuU0P4fK6qS/xjetW2Gmc1W8j//z5nAeNcC8Ttm0hV46IjcYeuVwYhuiw=="], + + "@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.1", "", {}, "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw=="], + + "@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], + + "@sveltejs/acorn-typescript": ["@sveltejs/acorn-typescript@1.0.13", "", { "peerDependencies": { "acorn": "^8.9.0" } }, "sha512-wgKggnhZVL9Bfx1OaKKTrYY9BFRk6C8UAkQNUcIv1+llzYrIqy+RZm5HPKzn0NpEBvTVhTqB4kQyllZywsRBRQ=="], + + "@sveltejs/adapter-auto": ["@sveltejs/adapter-auto@7.0.1", "", { "peerDependencies": { "@sveltejs/kit": "^2.0.0" } }, "sha512-dvuPm1E7M9NI/+canIQ6KKQDU2AkEefEZ2Dp7cY6uKoPq9Z/PhOXABe526UdW2mN986gjVkuSLkOYIBnS/M2LQ=="], + + "@sveltejs/kit": ["@sveltejs/kit@2.70.3", "", { "dependencies": { "@standard-schema/spec": "^1.0.0", "@sveltejs/acorn-typescript": "^1.0.9", "@types/cookie": "^0.6.0", "acorn": "^8.16.0", "cookie": "^0.6.0", "devalue": "^5.8.1", "esm-env": "^1.2.2", "kleur": "^4.1.5", "magic-string": "^0.30.5", "mrmime": "^2.0.0", "set-cookie-parser": "^3.0.0", "sirv": "^3.0.0" }, "peerDependencies": { "@opentelemetry/api": "^1.0.0", "@sveltejs/vite-plugin-svelte": "^3.0.0 || ^4.0.0-next.1 || ^5.0.0 || ^6.0.0-next.0 || ^7.0.0", "svelte": "^4.0.0 || ^5.0.0-next.0", "typescript": "^5.3.3 || ^6.0.0", "vite": "^5.0.3 || ^6.0.0 || ^7.0.0-beta.0 || ^8.0.0" }, "optionalPeers": ["@opentelemetry/api", "typescript"], "bin": { "svelte-kit": "svelte-kit.js" } }, "sha512-UDvEYuZqAMbfB/oXIoqKvbKcb7YczK5zYrzmsGV1zRJk03jntwp8dXiYoIJotxAndsKvcPFtx9H1GRSKFdSHgg=="], + + "@sveltejs/load-config": ["@sveltejs/load-config@0.2.3", "", {}, "sha512-VT3qmUb8pRV2QrZjd8iAmtg8lf4W0TIjZbvXtz5MKei/q96teWZgGJyyidJzOjzZzvdq616eSRVeMYIQChUTAQ=="], + + "@sveltejs/vite-plugin-svelte": ["@sveltejs/vite-plugin-svelte@7.3.0", "", { "dependencies": { "deepmerge": "^4.3.1", "magic-string": "^1.0.0", "obug": "^2.1.0", "vitefu": "^1.1.2" }, "peerDependencies": { "svelte": "^5.46.4", "vite": "^8.0.0-beta.7 || ^8.0.0" } }, "sha512-QbRoJyD92e9R0ufeQIWRHrCC0ObcqSv/aBDdrQMoU+sypav3cDx5wytdQ6GLdXjEMO6xjrXGzfkUygng8JMv0A=="], + + "@types/cookie": ["@types/cookie@0.6.0", "", {}, "sha512-4Kh9a6B2bQciAhf7FSuMRRkUWecJgJu9nPnx3yzpsfXX/c50REIqpHY4C82bXP90qrLtXtkDxTZosYO3UpOwlA=="], + + "@types/estree": ["@types/estree@1.0.9", "", {}, "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg=="], + + "acorn": ["acorn@8.18.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-lGq+9yr1/GuAWaVYIHRjvvySG5/4VfKIvC8EWxStPdcDh/Ka7FG3twP6v4d5BkravUilhIAsG4Qj83t02LWUPQ=="], + + "aria-query": ["aria-query@5.3.1", "", {}, "sha512-Z/ZeOgVl7bcSYZ/u/rh0fOpvEpq//LZmdbkXyc7syVzjPAhfOa9ebsdTSjEBDU4vs5nC98Kfduj1uFo0qyET3g=="], + + "axobject-query": ["axobject-query@4.1.0", "", {}, "sha512-qIj0G9wZbMGNLjLmg1PT6v2mE9AH2zlnADJD/2tC6E00hgmhUOfEB6greHPAfLRSufHqROIUTkw6E+M3lH0PTQ=="], + + "chokidar": ["chokidar@4.0.3", "", { "dependencies": { "readdirp": "^4.0.1" } }, "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA=="], + + "clsx": ["clsx@2.1.1", "", {}, "sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA=="], + + "cookie": ["cookie@0.6.0", "", {}, "sha512-U71cyTamuh1CRNCfpGY6to28lxvNwPG4Guz/EVjgf3Jmzv0vlDp1atT9eS5dDjMYHucpHbWns6Lwf3BKz6svdw=="], + + "deepmerge": ["deepmerge@4.3.1", "", {}, "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="], + + "detect-libc": ["detect-libc@2.1.2", "", {}, "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ=="], + + "devalue": ["devalue@5.9.2", "", {}, "sha512-po4PAY5c53tw5XMocSnf8A/5OHhbbUftpr93aEN6BBoAdntUmK7vu7wOATqvt7cXO7m1Cl4gMVn6p7n6n4mj0w=="], + + "esm-env": ["esm-env@1.2.2", "", {}, "sha512-Epxrv+Nr/CaL4ZcFGPJIYLWFom+YeV1DqMLHJoEd9SYRxNbaFruBwfEX/kkHUJf55j2+TUbmDcmuilbP1TmXHA=="], + + "esrap": ["esrap@2.3.7", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.4.15" }, "peerDependencies": { "@typescript-eslint/types": "^8.2.0" }, "optionalPeers": ["@typescript-eslint/types"] }, "sha512-n2nf7fZR3c9yXf0BPEuHuXqT+KW0SJVj4cN5FMEkpCZ3scLjOQWpiccyCxVzCC2q1wubTghuEGzngJY/7Ah0Ow=="], + + "fdir": ["fdir@6.5.0", "", { "peerDependencies": { "picomatch": "^3 || ^4" }, "optionalPeers": ["picomatch"] }, "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg=="], + + "fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="], + + "is-reference": ["is-reference@3.0.3", "", { "dependencies": { "@types/estree": "^1.0.6" } }, "sha512-ixkJoqQvAP88E6wLydLGGqCJsrFUnqoH6HnaczB8XmDH1oaWU+xxdptvikTgaEhtZ53Ky6YXiBuUI2WXLMCwjw=="], + + "kleur": ["kleur@4.1.5", "", {}, "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ=="], + + "lightningcss": ["lightningcss@1.33.0", "", { "dependencies": { "detect-libc": "^2.0.3" }, "optionalDependencies": { "lightningcss-android-arm64": "1.33.0", "lightningcss-darwin-arm64": "1.33.0", "lightningcss-darwin-x64": "1.33.0", "lightningcss-freebsd-x64": "1.33.0", "lightningcss-linux-arm-gnueabihf": "1.33.0", "lightningcss-linux-arm64-gnu": "1.33.0", "lightningcss-linux-arm64-musl": "1.33.0", "lightningcss-linux-x64-gnu": "1.33.0", "lightningcss-linux-x64-musl": "1.33.0", "lightningcss-win32-arm64-msvc": "1.33.0", "lightningcss-win32-x64-msvc": "1.33.0" } }, "sha512-WkUDrojuJs0xkgGf2udWxa3yGBRxPtxUkB79i6aCZLRgc7PM8fZe9TosfPDcvEpQZbuFASnHYmRLBLUbmLOIIA=="], + + "lightningcss-android-arm64": ["lightningcss-android-arm64@1.33.0", "", { "os": "android", "cpu": "arm64" }, "sha512-gEpRTalKdosp4Bb8qWtc2iOgE5SeIHlpS1up9bFq2wAyYhl1UdTObYiHe98zEM9SQvSoqQZ1IQD0JNpg3Ml5pg=="], + + "lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.33.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-Sciaz8eenNTKn9b3t7+xr0ipTp9YxKQY4npwQ3mrRuL0BAVHBLyZxofhaKBAVtzmtRZ/zTyo0/to4B1uWG/Djg=="], + + "lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.33.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-Z5UPAxzrjlWNNyGy6i65cJzzvgJ5D3T6wMvs+gWpY9d7qRhANrxqAp6LhxIgZhWEw18RfJTGcRxjuLIBr+m8XQ=="], + + "lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.33.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-QQM/Ti/hQajJwCY+RiWuCZ9sdtI/XQk7nDK5vC8kkdwixezOlDgvDx7+RT+QjK6FcFT4MpsuoBnHIo/O3StRRg=="], + + "lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.33.0", "", { "os": "linux", "cpu": "arm" }, "sha512-N7FVBe6iS24MlM6R/4RBTxGhQheZGs7tiQ9U32UtF75NzP5Q7xWPRqLBCKxlRQRk3rY1jCIPLzx7WzOhuUIRLQ=="], + + "lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.33.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-j2v/itmy4HlNxlc6voKXYgBqNi0Ng2LShg4z7GufpEgs05P+2suBVyi9I6YHq5uoVFx9ETin3eCEhLVyXGQnKg=="], + + "lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.33.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-yiO5ROMuYQgXbC60yjZU5CYSFZGKXL0HFATXt9mHJn1+zW55oCtMI9NfcVhYLMFDL7gV7oBPon/EmMMGg2OvtQ=="], + + "lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.33.0", "", { "os": "linux", "cpu": "x64" }, "sha512-ar+Ju7LmcN0Jo4FpL4hpFybwNG9/3A/Br5KW2n2jyODg3MEZXaDYADdemoNS+BDNfMgKvylJLj4S5tyRActuAg=="], + + "lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.33.0", "", { "os": "linux", "cpu": "x64" }, "sha512-RYiYbkokw0trfKqqzfF55lginwEPrD3OJDfTuJzFs1MK6iFnDenaz1fqLLtX4ITG3OktJQXOeTaw1awrBAlZPw=="], + + "lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.33.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-1K+MPfLSFVpphzpdbfkhlWk6wBrTObBzS2T6db10PNOZgR9GoVsAWzwNyuhUYYbTp23j+4RrncfujZ4uAzXvwA=="], + + "lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.33.0", "", { "os": "win32", "cpu": "x64" }, "sha512-OlEICDx/Xl0FqSp4bry8zFnCvGpig3Gl4gCquvYwHuqJKEC1+n9NgDniFvqHGmMv1ZkqDJrDqKKSykTDX+ehuA=="], + + "locate-character": ["locate-character@3.0.0", "", {}, "sha512-SW13ws7BjaeJ6p7Q6CO2nchbYEc3X3J6WrmTTDto7yMPqVSZTUyY5Tjbid+Ab8gLnATtygYtiDIJGQRRn2ZOiA=="], + + "magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="], + + "mri": ["mri@1.2.0", "", {}, "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA=="], + + "mrmime": ["mrmime@2.0.1", "", {}, "sha512-Y3wQdFg2Va6etvQ5I82yUhGdsKrcYox6p7FfL1LbK2J4V01F9TGlepTIhnK24t7koZibmg82KGglhA1XK5IsLQ=="], + + "nanoid": ["nanoid@3.3.18", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w=="], + + "obug": ["obug@2.1.4", "", {}, "sha512-4a+OsYv9UktOJKE+l1A4OufDgdRF9PifWj+tJnHURo/P+WOxpG4GzUFL9qCalmWauao6ogiG+QvnCovwPoyAWA=="], + + "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="], + + "picomatch": ["picomatch@4.0.7", "", {}, "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA=="], + + "postcss": ["postcss@8.5.28", "", { "dependencies": { "nanoid": "^3.3.18", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" } }, "sha512-RRuzqDtt5Y9h3quz5hWhK+TPnsmVs6WwSU6LkJMeY4HstUEDuYTG8UJSdawMRzmzAtV+KEoG8N3Qg2qLy5vM/A=="], + + "readdirp": ["readdirp@4.1.2", "", {}, "sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg=="], + + "rolldown": ["rolldown@1.2.7", "", { "dependencies": { "@oxc-project/types": "=0.148.0", "@rolldown/pluginutils": "^1.0.0" }, "optionalDependencies": { "@rolldown/binding-android-arm-eabi": "1.2.7", "@rolldown/binding-android-arm64": "1.2.7", "@rolldown/binding-darwin-arm64": "1.2.7", "@rolldown/binding-darwin-x64": "1.2.7", "@rolldown/binding-freebsd-x64": "1.2.7", "@rolldown/binding-linux-arm-gnueabihf": "1.2.7", "@rolldown/binding-linux-arm64-gnu": "1.2.7", "@rolldown/binding-linux-arm64-musl": "1.2.7", "@rolldown/binding-linux-ppc64-gnu": "1.2.7", "@rolldown/binding-linux-s390x-gnu": "1.2.7", "@rolldown/binding-linux-x64-gnu": "1.2.7", "@rolldown/binding-linux-x64-musl": "1.2.7", "@rolldown/binding-openharmony-arm64": "1.2.7", "@rolldown/binding-win32-arm64-msvc": "1.2.7", "@rolldown/binding-win32-x64-msvc": "1.2.7" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-g0EtLvBjTUB7jhyV0S/TCup3v/XSVl45vUIGbOGU4QPiyjTenCe4mKuFvW9fEgYmS2Fo42AUssRmNuMziXdrig=="], + + "sade": ["sade@1.8.1", "", { "dependencies": { "mri": "^1.1.0" } }, "sha512-xal3CZX1Xlo/k4ApwCFrHVACi9fBqJ7V+mwhBsuf/1IOKbBy098Fex+Wa/5QMubw09pSZ/u8EY8PWgevJsXp1A=="], + + "set-cookie-parser": ["set-cookie-parser@3.1.2", "", {}, "sha512-5/r/lTwbJ3zQ+qwdUFZYeRNqda7P5HD8zQKqlSjdGt1/S0cjLAphHusj4Y58ahDtWn/g32xrIS58/ikOvwl0Lw=="], + + "sirv": ["sirv@3.0.2", "", { "dependencies": { "@polka/url": "^1.0.0-next.24", "mrmime": "^2.0.0", "totalist": "^3.0.0" } }, "sha512-2wcC/oGxHis/BoHkkPwldgiPSYcpZK3JU28WoMVv55yHJgcZ8rlXvuG9iZggz+sU1d4bRgIGASwyWqjxu3FM0g=="], + + "source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="], + + "svelte": ["svelte@5.57.0", "", { "dependencies": { "@jridgewell/remapping": "^2.3.4", "@jridgewell/sourcemap-codec": "^1.5.0", "@sveltejs/acorn-typescript": "^1.0.10", "@types/estree": "^1.0.5", "acorn": "^8.12.1", "aria-query": "5.3.1", "axobject-query": "^4.1.0", "clsx": "^2.1.1", "devalue": "^5.8.1", "esm-env": "^1.2.1", "esrap": "^2.2.12", "is-reference": "^3.0.3", "locate-character": "^3.0.0", "magic-string": "^0.30.11", "zimmerframe": "^1.1.2" } }, "sha512-NdbDn7fl4be1ViUG0oq/lvG6OZy3oENolV2ONjiqqsfVoeAfzaQAKUcEX3MrQod/Bebv1PgwET9rfXhgn9s4Kg=="], + + "svelte-check": ["svelte-check@4.7.6", "", { "dependencies": { "@jridgewell/trace-mapping": "^0.3.25", "@sveltejs/load-config": "^0.2.3", "chokidar": "^4.0.1", "fdir": "^6.2.0", "picocolors": "^1.0.0", "sade": "^1.7.4" }, "peerDependencies": { "svelte": "^4.0.0 || ^5.0.0-next.0", "typescript": "^5.0.0 || ^6.0.0" }, "bin": { "svelte-check": "bin/svelte-check" } }, "sha512-t2scM//ZuVbSY/T2w6FSBw1v9s2NEmh/g+sy1lqtosW5ylBV5AF4wFb1Ts9Kf3MbfPDUDJDZ9L436YT0SPTdvw=="], + + "tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="], + + "totalist": ["totalist@3.0.1", "", {}, "sha512-sf4i37nQ2LBx4m3wB74y+ubopq6W/dIzXg0FDGjsYnZHVa1Da8FH853wlL2gtUhg+xJXjfk3kUZS3BRoQeoQBQ=="], + + "typescript": ["typescript@6.0.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw=="], + + "vite": ["vite@8.2.2", "", { "dependencies": { "lightningcss": "^1.33.0", "picomatch": "^4.0.5", "postcss": "^8.5.26", "rolldown": "~1.2.4", "tinyglobby": "^0.2.17" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.4.0 || ^0.5.0", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q=="], + + "vitefu": ["vitefu@1.1.3", "", { "peerDependencies": { "vite": "^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 || ^7.0.0 || ^8.0.0" }, "optionalPeers": ["vite"] }, "sha512-ub4okH7Z5KLjb6hDyjqrGXqWtWvoYdU3IGm/NorpgHncKoLTCfRIbvlhBm7r0YstIaQRYlp4yEbFqDcKSzXSSg=="], + + "zimmerframe": ["zimmerframe@1.1.5", "", {}, "sha512-msJxIvYDYcoNL+PJsu+7qmpDWsYmAxTY+2TNYXXF0hzBzBk0BMecOqDOG/EckUoKCuKwObfbugIl8QpqHDXeFA=="], + + "@sveltejs/vite-plugin-svelte/magic-string": ["magic-string@1.2.3", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-Bpb0W2TbLKOZ7vJnOUnVRGq3WL2p+ISV29M6hYPL1AFCpyKZpdr5ytiXoTSSxRVhg8YW7f65+6gbG8WG6PCa/g=="], + } +} diff --git a/indexium-frontend/package.json b/indexium-frontend/package.json new file mode 100644 index 0000000..9aa85e9 --- /dev/null +++ b/indexium-frontend/package.json @@ -0,0 +1,23 @@ +{ + "name": "indexium-frontend", + "private": true, + "version": "0.0.1", + "type": "module", + "scripts": { + "dev": "vite dev", + "build": "vite build", + "preview": "vite preview", + "prepare": "svelte-kit sync || echo ''", + "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", + "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch" + }, + "devDependencies": { + "@sveltejs/adapter-auto": "^7.0.1", + "@sveltejs/kit": "^2.63.0", + "@sveltejs/vite-plugin-svelte": "^7.1.2", + "svelte": "^5.56.1", + "svelte-check": "^4.6.0", + "typescript": "^6.0.3", + "vite": "^8.0.16" + } +} diff --git a/indexium-frontend/src/app.d.ts b/indexium-frontend/src/app.d.ts new file mode 100644 index 0000000..da08e6d --- /dev/null +++ b/indexium-frontend/src/app.d.ts @@ -0,0 +1,13 @@ +// See https://svelte.dev/docs/kit/types#app.d.ts +// for information about these interfaces +declare global { + namespace App { + // interface Error {} + // interface Locals {} + // interface PageData {} + // interface PageState {} + // interface Platform {} + } +} + +export {}; diff --git a/indexium-frontend/src/app.html b/indexium-frontend/src/app.html new file mode 100644 index 0000000..6a2bb58 --- /dev/null +++ b/indexium-frontend/src/app.html @@ -0,0 +1,12 @@ + + + + + + + %sveltekit.head% + + +
%sveltekit.body%
+ + diff --git a/indexium-frontend/src/lib/api.ts b/indexium-frontend/src/lib/api.ts new file mode 100644 index 0000000..2a569b1 --- /dev/null +++ b/indexium-frontend/src/lib/api.ts @@ -0,0 +1,194 @@ +/** + * Typed fetch client for Indexium Public API v1. + * Base URL via VITE_PUBLIC_API_URL (Vite / SvelteKit client env). + */ + +export const BASE: string = + (import.meta.env.VITE_PUBLIC_API_URL as string | undefined) ?? 'http://localhost:8080'; + +// --------------------------------------------------------------------------- +// Error +// --------------------------------------------------------------------------- + +export class ApiError extends Error { + status: number; + constructor(status: number, message: string) { + super(message); + this.name = 'ApiError'; + this.status = status; + } +} + +// --------------------------------------------------------------------------- +// Types (mirror api-spec.md + backend DTOs) +// --------------------------------------------------------------------------- + +export interface ModListItem { + slug: string; + name: string; + summary: string | null; + author: string; + icon_url: string | null; + game_versions: string[]; + loaders: string[]; + latest_version: string | null; + download_url: string | null; + updated_at: string; +} + +export interface Pagination { + page: number; + limit: number; + total: number; + pages: number; +} + +export interface ModListResponse { + data: ModListItem[]; + pagination: Pagination; +} + +export interface AuthorDto { + login: string; + avatar_url: string | null; +} + +export interface VersionDto { + version_number: string; + game_versions: string[]; + loaders: string[]; + download_url: string; + file_sha256: string; + file_size: number | null; + published_at: string; +} + +export interface ModDetail { + slug: string; + name: string; + summary: string | null; + description: string | null; + github_repo: string; + author: AuthorDto; + icon_url: string | null; + verified: boolean; + versions: VersionDto[]; + updated_at: string; +} + +export interface DailyPoint { + date: string; + active_servers: number; + active_players: number; +} + +export interface Breakdown { + mc_versions: Record; + loaders: Record; + os: Record; + java: Record; + custom: Record>; +} + +export interface AnalyticsResponse { + mod_slug: string; + range: string; + daily: DailyPoint[]; + breakdown: Breakdown; +} + +// --------------------------------------------------------------------------- +// Params +// --------------------------------------------------------------------------- + +export interface FetchModsParams { + query?: string; + gameVersion?: string; + loader?: string; + page?: number; + limit?: number; + sort?: 'relevance' | 'newest' | 'popular' | 'active_servers'; +} + +export type AnalyticsRange = '7d' | '30d' | '90d'; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Build querystring, skipping empty / undefined values. */ +export function buildQuery(params: Record): string { + const sp = new URLSearchParams(); + for (const [k, v] of Object.entries(params)) { + if (v === undefined || v === null || v === '') continue; + sp.set(k, String(v)); + } + const qs = sp.toString(); + return qs ? `?${qs}` : ''; +} + +async function request(path: string, init?: RequestInit): Promise { + const url = `${BASE}${path}`; + const res = await fetch(url, { + headers: { 'Content-Type': 'application/json', ...(init?.headers ?? {}) }, + ...init + }); + if (!res.ok) { + let message = res.statusText; + try { + const body = (await res.json()) as { message?: string; error?: string }; + message = body.message ?? body.error ?? message; + } catch { + // ignore json parse error + } + throw new ApiError(res.status, message); + } + return (await res.json()) as T; +} + +// --------------------------------------------------------------------------- +// API functions +// --------------------------------------------------------------------------- + +export async function fetchMods(params: FetchModsParams = {}): Promise { + const qs = buildQuery({ + query: params.query, + gameVersion: params.gameVersion, + loader: params.loader, + page: params.page, + limit: params.limit, + sort: params.sort + }); + return request(`/api/v1/mods${qs}`); +} + +export async function fetchMod(slug: string): Promise { + return request(`/api/v1/mods/${encodeURIComponent(slug)}`); +} + +export async function fetchAnalytics( + slug: string, + range: AnalyticsRange = '30d' +): Promise { + const qs = buildQuery({ range }); + return request(`/api/v1/mods/${encodeURIComponent(slug)}/analytics${qs}`); +} + +/** Stub for analytics ingestion (mod SDK → POST /analytics/submit). */ +export async function submitAnalytics(payload: { + mod_slug: string; + server_uuid: string; + metrics: { + mc_version: string; + loader: string; + java_version: string; + os: string; + player_count: number; + custom_charts?: Record; + }; +}): Promise<{ status: string }> { + return request<{ status: string }>('/api/v1/analytics/submit', { + method: 'POST', + body: JSON.stringify(payload) + }); +} diff --git a/indexium-frontend/src/lib/assets/favicon.svg b/indexium-frontend/src/lib/assets/favicon.svg new file mode 100644 index 0000000..cc5dc66 --- /dev/null +++ b/indexium-frontend/src/lib/assets/favicon.svg @@ -0,0 +1 @@ +svelte-logo \ No newline at end of file diff --git a/indexium-frontend/src/lib/index.ts b/indexium-frontend/src/lib/index.ts new file mode 100644 index 0000000..856f2b6 --- /dev/null +++ b/indexium-frontend/src/lib/index.ts @@ -0,0 +1 @@ +// place files you want to import through the `$lib` alias in this folder. diff --git a/indexium-frontend/src/routes/+layout.svelte b/indexium-frontend/src/routes/+layout.svelte new file mode 100644 index 0000000..9cebde5 --- /dev/null +++ b/indexium-frontend/src/routes/+layout.svelte @@ -0,0 +1,11 @@ + + + + + + +{@render children()} diff --git a/indexium-frontend/src/routes/+page.svelte b/indexium-frontend/src/routes/+page.svelte new file mode 100644 index 0000000..dfe95e5 --- /dev/null +++ b/indexium-frontend/src/routes/+page.svelte @@ -0,0 +1,122 @@ + + + + Indexium — Mods + + +
+

Indexium — Minecraft Mods

+

Лёгкий индекс модов поверх GitHub Releases CDN

+ +
+ {#each loaders as l} + + {/each} +
+ + {#if loading} +

Loading mods…

+ {:else if error} +
+ Ошибка: + {error} + +
+ {:else if mods.length === 0} +

Модов не найдено.

+ {:else} + {#if pagination} +

+ Найдено {pagination.total} · страница {pagination.page} из {pagination.pages} +

+ {/if} +
+ {#each mods as mod (mod.slug)} +
+

{mod.name}

+

+ {mod.summary ?? 'Без описания'} +

+
+ {#each mod.loaders as loader} + {loader} + {/each} + {#each mod.game_versions as gv} + {gv} + {/each} +
+
+ {mod.slug} + Подробнее → +
+
+ {/each} +
+ {/if} +
diff --git a/indexium-frontend/static/robots.txt b/indexium-frontend/static/robots.txt new file mode 100644 index 0000000..b6dd667 --- /dev/null +++ b/indexium-frontend/static/robots.txt @@ -0,0 +1,3 @@ +# allow crawling everything by default +User-agent: * +Disallow: diff --git a/indexium-frontend/tsconfig.json b/indexium-frontend/tsconfig.json new file mode 100644 index 0000000..2c2ed3c --- /dev/null +++ b/indexium-frontend/tsconfig.json @@ -0,0 +1,20 @@ +{ + "extends": "./.svelte-kit/tsconfig.json", + "compilerOptions": { + "rewriteRelativeImportExtensions": true, + "allowJs": true, + "checkJs": true, + "esModuleInterop": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "skipLibCheck": true, + "sourceMap": true, + "strict": true, + "moduleResolution": "bundler" + } + // Path aliases are handled by https://svelte.dev/docs/kit/configuration#alias + // except $lib which is handled by https://svelte.dev/docs/kit/configuration#files + // + // To make changes to top-level options such as include and exclude, we recommend extending + // the generated config; see https://svelte.dev/docs/kit/configuration#typescript +} diff --git a/indexium-frontend/vite.config.ts b/indexium-frontend/vite.config.ts new file mode 100644 index 0000000..cb76b81 --- /dev/null +++ b/indexium-frontend/vite.config.ts @@ -0,0 +1,20 @@ +import adapter from '@sveltejs/adapter-auto'; +import { sveltekit } from '@sveltejs/kit/vite'; +import { defineConfig } from 'vite'; + +export default defineConfig({ + plugins: [ + sveltekit({ + compilerOptions: { + // Force runes mode for the project, except for libraries. Can be removed in svelte 6. + runes: ({ filename }) => + filename.split(/[/\\]/).includes('node_modules') ? undefined : true + }, + + // adapter-auto only supports some environments, see https://svelte.dev/docs/kit/adapter-auto for a list. + // If your environment is not supported, or you settled on a specific environment, switch out the adapter. + // See https://svelte.dev/docs/kit/adapters for more information about adapters. + adapter: adapter() + }) + ] +}); diff --git a/todo.md b/todo.md new file mode 100644 index 0000000..0c23ba6 --- /dev/null +++ b/todo.md @@ -0,0 +1,180 @@ +# Indexium — TODO / Roadmap + +> Сервис: асинхронный событийный индексатор модов Minecraft поверх GitHub Releases CDN. +> Бэкенд не хранит тяжёлые артефакты, только метаданные + индексация + быстрый JSON API. + +--- + +## Правила проекта (обязательно к соблюдению) + +> Эти правила — не чекбоксы, а инварианты. Любой PR, нарушающий их, не принимается. + +### 1. KISS — Keep It Simple, Stupid +- Выбирай самое простое решение, которое закрывает задачу. Никаких абстракций «на будущее» (YAGNI). +- Один модуль — одна ответственность. Если не можешь объяснить функцию в одном предложении — дроби. +- Предпочитай явный код неявной магии (никаких макросов ради макросов). + +### 2. DRY — Don't Repeat Yourself +- Повтор >2 раз → выноси в функцию/модуль. Но не DRY ради DRY: дублирование лучше неправильной абстракции. +- Общие типы/утилиты — в `common`/`shared`, доменная логика — в своём модуле. + +### 3. SOLID (применительно к Rust) +- **S** — один файл/модуль = одна причина для изменений (см. лимиты ниже). +- **O** — открыт для расширения через трейты, закрыт для модификации (feature-flag, а не `if` на типы). +- **L** — любой `impl Trait` должен заменять другой без поломки контракта. +- **I** — узкие трейты лучше жирных (`Readable`, `Validatable` вместо `GodService`). +- **D** — зависимость от абстракций (`PgPool` через `AppState`, а не глобаль). + +### 4. Лимиты структуры (жёстко) +- **Макс 250 строк на файл** — если больше, дроби файл на подмодули. +- **Макс 4 файла на папку** — если больше, вводи подпапки по домену (`api/mods/`, `worker/parsers/`). +- Исключение: `mod.rs`/`lib.rs` не считаются, но должны быть тонкими реэкспортами. +- CI будет ругаться (`cargo clippy` + кастомный скрипт `scripts/check-limits.sh`). + +### 5. Дополнительные инварианты +- **Чистота ошибок**: никаких `unwrap()`/`expect()` вне `main.rs` и тестов. Везде `Result` + `thiserror`/`anyhow`. +- **Типы вместо строк**: `Slug`, `GameVersion`, `Loader` — newtype, а не `String`. +- **Миграции только вперёд**: никаких `DROP` без ADR и бэкапа. Каждая миграция — идемпотентна (`IF NOT EXISTS`). +- **Логика без сайд-эффектов**: парсеры/валидаторы — чистые функции, I/O только на границах (handler/worker). +- **Документация рядом с кодом**: публичная функция без `///` — не готова к мерджу. +- **Тест на каждый баг**: регрессия покрывается тестом до фикса. + +--- + +## Легенда статусов +- `[ ]` — не начато +- `[~]` — в процессе +- `[x]` — готово +- `[!]` — заблокировано / требует решения + +--- + +## Phase 0 — Фундамент монорепо (Текущий приоритет) + +- [x] Объединить `indexium-backend` + `indexium-frontend` в один git-монорепо (корень `/`) +- [x] Настроить корневой `.gitignore` + локальные `.gitignore` +- [x] Создать структуру `docs/` и заполнить базовую архитектуру +- [x] Создать `todo.md` и `README.md` в корне + прописать правила KISS/DRY/SOLID и лимиты файлов +- [x] Первая миграция `20260906000000_init_schema.sql` (mods, mod_versions, GIN индексы) +- [x] Базовый `src/main.rs` (Axum + SQLx + CORS + /health + миграции при старте) +- [x] Настройка `.env` / `.env.example` (DATABASE_URL, SERVER_PORT) +- [ ] Добавить `docker-compose.yml` (Postgres + Redis/Valkey) для локальной разработки +- [ ] Добавить `Makefile` / `justfile` с командами `dev`, `migrate`, `lint`, `test` +- [ ] Настроить CI (GitHub Actions): `cargo clippy + test`, `svelte-check`, `sqlx migrate check` +- [ ] Скрипт `scripts/check-limits.sh` — проверка 250 строк / 4 файла на папку + +## Phase 1 — Backend Core (Rust / Axum) + +### 1.1 Инфраструктура +- [x] `config` — загрузка `.env` (DATABASE_URL, REDIS_URL, GITHUB_APP_ID, WEBHOOK_SECRET) — база в `main.rs` через `dotenvy` +- [x] `db` — пул `sqlx::PgPool`, миграции (`sqlx::migrate!`), health-check `/health` +- [x] `tracing` — структурированные логи (EnvFilter + fmt layer) +- [x] Axum роутер: `GET /health`, CORS (5173), TraceLayer + +### 1.2 Схема БД (PostgreSQL + FTS + pg_trgm) +- [x] Миграция `20260906000000_init_schema.sql` — таблицы `mods`, `mod_versions` + `GIN (game_versions, loaders)` +- [ ] Миграция `002_fts` — `search_vector`, `pg_trgm`, триггер (см. `docs/database-schema.md`) +- [ ] Таблица `authors` + `webhook_deliveries` (идемпотентность) +- [ ] Сиды / фикстуры для локального дев-окружения + +### 1.3 Webhook Ingestion API +- [ ] `POST /api/v1/webhooks/github` — проверка `X-Hub-Signature-256` (HMAC SHA-256) +- [ ] Валидация эвента `release.published` / `release.released`, идемпотентность по `delivery_id` +- [ ] Пуш задачи в очередь (Redis Streams) + ответ `202 Accepted` < 50ms +- [ ] Тест на replay-атаку и неверную подпись + +### 1.4 Async Worker / Indexer +- [ ] Консьюмер очереди (tokio task) +- [ ] Скачивание через HTTP Range Request — чтение только ZIP central directory `.jar` +- [ ] Парсинг `fabric.mod.json` / `quilt.mod.json` / `neoforge.mods.toml` / `mcmod.info` +- [ ] Валидация: `mod_id`, `version`, `game_versions`, `loaders`, иконка +- [ ] SHA-256 сверка (если приложен `.sha256`), отбраковка битого артефакта +- [ ] Бейсик malware-скан: поиск `Runtime.exec`, `URLClassLoader`, сетевых вызовов в `` +- [ ] Сохранение в `mod_versions`, инвалидация Redis-кэша + +### 1.5 Public REST API (Read-Heavy, Cache-First) +- [ ] `GET /api/v1/mods?query=&gameVersion=&loader=&page=&limit=` — FTS + фильтры, кэш Redis 60s +- [ ] `GET /api/v1/mods/:slug` — карточка мода + список версий +- [ ] `GET /api/v1/mods/:slug/versions/:version` — детали версии + `download_url` (прямая CDN ссылка GitHub) +- [ ] Пагинация cursor/offset, ETag, `Cache-Control` +- [ ] Rate limiting (tower_governor / redis-cell) + +### 1.6 Auth & Profiles (см. docs/auth-profiles.md, adr/004) +- [ ] GitHub OAuth 2.0 (read:user, user:email) + JWT httpOnly — вход для авторов (MVP) +- [ ] PAT `personal_access_tokens` (hash, scopes, expires) — `POST /auth/tokens` для CLI/лаунчеров (MVP) +- [ ] `POST /api/v1/mods/import` — импорт репозитория (проверка LICENSE + public + манифест) +- [ ] Профили `/u/:login`, `/org/:login` — кэш ISR, sponsors, verified badge, SVG `/v1/badges/:slug/*.svg` (MVP) +- [ ] Star/Follow `stars`, `follows` (с фильтром game_version/loader) — in-app уведомления (MVP-лайт) +- [ ] Device Flow RFC8628 (`/oauth/device/code` → `/activate`) — спроектировать, реализация Phase 2 +- [ ] Discord linked_accounts + бот роли Verified Modder — Phase 2 +- [ ] Установка Webhook'а через GitHub App API (автоматически) — миграция с OAuth на App в Phase 2 + +## Phase 2 — Frontend (SvelteKit) + Social + +- [ ] Дизайн-система: Tailwind / UnoCSS + токены +- [ ] Страницы: `/` (поиск + фильтры), `/mod/[slug]`, `/mods/import`, `/u/[login]`, `/org/[login]`, `/activate` (device flow) +- [ ] Компоненты: `ModCard`, `VersionTable`, `SearchBar`, `LoaderBadge`, `ProfileHeader`, `SponsorsBar` +- [ ] Клиент API (`src/lib/api.ts`) — типизированные fetch-обёртки +- [ ] SSR + кэширование, skeletons, error boundaries +- [ ] SEO / OpenGraph для карточек модов +- [ ] **Collections / Modlists** `collections`, `collection_stars` + экспорт `?format=prism|packwiz` (Phase 2 хит) +- [ ] **Activity Feed** — лента по подпискам (releases + collections + stars) +- [ ] **Org/Teams** — `/org/:login` агрегатор, `role=maintainer` +- [ ] **Геймификация** — `badges` (Early Adopter, Bug Hunter, Veteran) + Showcase SVG + +## Phase 3 — Поиск, качество данных и аналитика (см. docs/analytics.md, adr/005) + +- [ ] PostgreSQL FTS (`to_tsvector` + `ts_rank`) по `name`, `summary`, `README` +- [ ] `pg_trgm` для неточных совпадений / опечаток +- [ ] Опционально: `pgvector` для семантического поиска по описанию (эмбеддинги README) +- [ ] Админ-панель / ручная модерация, флаг `verified` / `suspicious` +- [ ] **Indexium Analytics (bStats аналог):** + - [ ] Миграция `20260907000000_telemetry.sql` (`mod_telemetry_pings`, `mod_daily_stats`, `analytics_salts`) + - [ ] `POST /api/v1/analytics/submit` (gzip, валидация, daily_salt hash, Redis 1/15мин, без IP) + - [ ] Крон агрегация `COUNT(DISTINCT server_hash)` → `mod_daily_stats` + TTL 30д + - [ ] `GET /mods/:slug/analytics?range=30d` + `GET /badges/:slug/servers.svg` + `sort=active_servers` + - [ ] Легковесный Java/Kotlin SDK `dev.indexium:analytics` (MIT, SimplePie, opt-out флаг) + +## Phase 4 — Надёжность и ограничения GitHub + +- [ ] GitHub App Install token — 5k-12.5k RPH вместо 60 RPH анонимных +- [ ] Прямые редиректы на `objects.githubusercontent.com` — не проксировать трафик +- [ ] Retry + exponential backoff, DLQ для воркера +- [ ] Метрики: Prometheus / `tracing` + Grafana, алерты на lag очереди + +## Phase 5 — Деплой и эксплуатации + +- [ ] Dockerfile multi-stage для backend (distroless / alpine) +- [ ] Dockerfile для frontend (adapter-node / adapter-static) +- [ ] `docker-compose.prod.yml` / Fly.io / Railway / Hetzner +- [ ] Бэкапы Postgres (PITR), миграции в CI +- [ ] Документация деплоя (`docs/deployment.md`) + +## Phase 6 — Расширения (Backlog) + +- [ ] Поддержка CurseForge / Modrinth как доп. источников (опционально) +- [ ] Webhooks для лаунчеров (подписка на обновления мода) +- [ ] CLI для авторов (`indexium publish`) +- [x] Аналитика рантайма — аналог bStats (спроектирована, см. docs/analytics.md) → реализация в Phase 3 +- [ ] Аналитика скачиваний (агрегация без хранения персоналки) + +--- + +## Ближайшие 3 шага (Next Actions) + +1. `docker-compose.yml` + первая миграция SQL +2. `POST /webhooks/github` с HMAC-проверкой и заглушкой очереди (in-memory channel) +3. `GET /mods` — мок-данные из БД + подключение фронта + +--- + +## Как отмечать прогресс + +- При завершении задачи ставь `[x]` и добавляй ссылку на PR/коммит: `[x] Задача (#12)` +- Если задача блочится — ставь `[!]` и опиши блокер в комментарии ниже. + +## Блокеры / Вопросы + +- [ ] Выбрать окончательно очередь: `Redis Streams` vs `NATS JetStream` vs `pg-queue` на старте? + - Рекомендация: стартовать с `Redis` (уже нужен как кэш) → мигрировать на NATS если нужен strict ordering. +- [ ] Где хостить Postgres на старте — Supabase / Neon / self-hosted?