SBI, can we please talk about your unoffical API.

    • Talion wrote:

      Hey everyone,

      Thanks for bringing up your concerns about the performance of the API. Considering that this API was always meant to be for internal use, and thus entirely undocumented, the sheer number of tools that were developed and serve community purposes is astounding.

      This widespread acceptance and distribution of tools latching onto that API is, at the moment, its downfall. The increase in player numbers, paired with the increase in numbers of tools, brought the infrastructure behind this API pretty much down to its knees. That doesn’t make you happy because your tools don’t work, it doesn’t make us happy, because our killboard doesn’t work. We’re currently investigating which attempts at improving the situation are the ones most likely to succeed. Throwing even more hardware at the situation is obviously the first step, will likely only buy some time - and is also something that’s not an instant implementation. However, we’re on it.

      Going beyond that, we will have to look at the fundamental architecture of the API itself to enable it to handle all the data from the hundreds of thousands of people playing the game, as well as the tools that our community, that you guys, develop and maintain.

      -Talion
      Hardware-wise:
      Make internal server then move logs over the internal network to some DRBD clustered server behind loadbalancer and put the cached API for use. That should solve the issues with handling too many requests resulting 502 timeouts.
      So main API server can process data that will move to fully cached cluster. Simple and easy to set up. LXD may be your ally here, with good loadbalancing you can rotate containers based on pps/cpu/anystat usage.

      We probably can live without some 6months old data. So to make it cheaper instead of big NVMe/SSD disks you could just use SAS or even go with NLSAS. Few TB should be enough for static files and all the events < 6months.


      There are better solutions but this one would work ootb while for example you can work on new API.

      Further API-related as @ItsNewbie said:

      1. Implement identity services.
      2. Implement Rate Limiting. Both unauthenticated and authenticated. Authenticated rating limiting can ensure that people dont abuse the API.
      3. Implement filters on the end points that allow pulling of both kill and death events. This will stop those that need to constantly poll chasing death events.
      discord.com/invite/syKykHg --- AOLoot Logger | Discord BOTs: RegearBOT, AOServerStatus, Doorkeeper, Killbot, Battlebot
    • @Talion

      We appreciate API being worked on, it seems less laggy now and events are more/less up to date BUT

      Player who got kicked from a guild 2 days ago is still in that guild according to AlbionOnline.com website/API.

      When can we expect to have updated info on official website regarding guild members?
      discord.com/invite/syKykHg --- AOLoot Logger | Discord BOTs: RegearBOT, AOServerStatus, Doorkeeper, Killbot, Battlebot
    • I'm a developer and I work with API's daily. Even "unofficial" API's.

      Just because you can post and pull from a website/game without an official pathway does not mean it was intended and most of the time pisses people off. (sometimes they say "oh damn good job")

      You don't know how many times someone said "Oh, we didn't know you were using that, that's intended to be internal" and then they kill that shit in 2 hours because it's causing problems (the fact they didn't know about it is slightly scary).

      Here is the real problem in my opinion with an API from Albion, even this "unofficial" api. - (I'm positive there are already people using scripts to pull the data anyway with certain wire tools *cough*, but that requires a little manual work)

      - If you expose an API with instant updates from the marketplace, it would be severely detrimental to the market. People who have access, or have the know how, could literally destroy markets very quickly. Hell I would. I'd throw it in Power BI and start filling my own database with all of their data. In just a few hours I could easily see days, times, seconds, cities and much much more of when and where an item is sold. You could even start using it to predict market changes.
      - You already have excel heros that are doing this anyway, and raking in billions. It's just takes more work. I applaud them for that work.

      Don't get me wrong. I would LOVE it, I'd make so many things. I would have so much fun diving into the data.

      That does not mean I don't think it would make the game worse.

      Just my two cents.
    • alancode wrote:

      I'm a developer and I work with API's daily. Even "unofficial" API's.

      Just because you can post and pull from a website/game without an official pathway does not mean it was intended and most of the time pisses people off. (sometimes they say "oh damn good job")

      You don't know how many times someone said "Oh, we didn't know you were using that, that's intended to be internal" and then they kill that shit in 2 hours because it's causing problems (the fact they didn't know about it is slightly scary).

      Here is the real problem in my opinion with an API from Albion, even this "unofficial" api. - (I'm positive there are already people using scripts to pull the data anyway with certain wire tools *cough*, but that requires a little manual work)

      - If you expose an API with instant updates from the marketplace, it would be severely detrimental to the market. People who have access, or have the know how, could literally destroy markets very quickly. Hell I would. I'd throw it in Power BI and start filling my own database with all of their data. In just a few hours I could easily see days, times, seconds, cities and much much more of when and where an item is sold. You could even start using it to predict market changes.
      - You already have excel heros that are doing this anyway, and raking in billions. It's just takes more work. I applaud them for that work.

      Don't get me wrong. I would LOVE it, I'd make so many things. I would have so much fun diving into the data.

      That does not mean I don't think it would make the game worse.

      Just my two cents.
      I wonder how EVE has full market API and the market isn't destroyed ? Honest question here.
    • sh1ny wrote:

      alancode wrote:

      I'm a developer and I work with API's daily. Even "unofficial" API's.

      Just because you can post and pull from a website/game without an official pathway does not mean it was intended and most of the time pisses people off. (sometimes they say "oh damn good job")

      You don't know how many times someone said "Oh, we didn't know you were using that, that's intended to be internal" and then they kill that shit in 2 hours because it's causing problems (the fact they didn't know about it is slightly scary).

      Here is the real problem in my opinion with an API from Albion, even this "unofficial" api. - (I'm positive there are already people using scripts to pull the data anyway with certain wire tools *cough*, but that requires a little manual work)

      - If you expose an API with instant updates from the marketplace, it would be severely detrimental to the market. People who have access, or have the know how, could literally destroy markets very quickly. Hell I would. I'd throw it in Power BI and start filling my own database with all of their data. In just a few hours I could easily see days, times, seconds, cities and much much more of when and where an item is sold. You could even start using it to predict market changes.
      - You already have excel heros that are doing this anyway, and raking in billions. It's just takes more work. I applaud them for that work.

      Don't get me wrong. I would LOVE it, I'd make so many things. I would have so much fun diving into the data.

      That does not mean I don't think it would make the game worse.

      Just my two cents.
      I wonder how EVE has full market API and the market isn't destroyed ? Honest question here.
      I'm not sure, I've never played EVE. I can't have an opinion on that.
    • sh1ny wrote:

      alancode wrote:

      I'm a developer and I work with API's daily. Even "unofficial" API's.

      Just because you can post and pull from a website/game without an official pathway does not mean it was intended and most of the time pisses people off. (sometimes they say "oh damn good job")

      You don't know how many times someone said "Oh, we didn't know you were using that, that's intended to be internal" and then they kill that shit in 2 hours because it's causing problems (the fact they didn't know about it is slightly scary).

      Here is the real problem in my opinion with an API from Albion, even this "unofficial" api. - (I'm positive there are already people using scripts to pull the data anyway with certain wire tools *cough*, but that requires a little manual work)

      - If you expose an API with instant updates from the marketplace, it would be severely detrimental to the market. People who have access, or have the know how, could literally destroy markets very quickly. Hell I would. I'd throw it in Power BI and start filling my own database with all of their data. In just a few hours I could easily see days, times, seconds, cities and much much more of when and where an item is sold. You could even start using it to predict market changes.
      - You already have excel heros that are doing this anyway, and raking in billions. It's just takes more work. I applaud them for that work.

      Don't get me wrong. I would LOVE it, I'd make so many things. I would have so much fun diving into the data.

      That does not mean I don't think it would make the game worse.

      Just my two cents.
      I wonder how EVE has full market API and the market isn't destroyed ? Honest question here.
      Also, in some ways most people will just think it's normal. But more people will be billionaire.

      Maybe it has something to do with the fact that each market is different? If I knew WITHOUT having to move from city to city, what the margin of every item in the game was, I could easily purchase all the items and move them without having to do hours of research.
    • Well eve has sites like this:

      evemarketer.com

      API gives things like:



      And that game is around for 15+ years. Doesn't seem like the market is destroyed or anything and the information is available to anyone. Mind you, the difference is even bigger in eve - traveling from one region to another takes quite longer time than in here.

      I don't even know why people are complaining - real world trade is also full of sites and API's. You even have things like eToro that even allow you to follow bigger investors/traders and copy (automatically) what they do, to make money.
      Not trying to argue here that we should or should not have an API - i am fine with things as they are, i just don't think the "Oh noes API's will destroy the market/game" have any grounds.