Data Feed Manager for Magento 2: User Guide

Data Feed Manager exports your products as XML, JSON, CSV or delimited TXT feeds for Google Shopping, Meta (Facebook and Instagram), Pinterest, Microsoft/Bing Shopping, price comparison sites and ERPs. Feeds are generated in batches by cron, from the admin or on the command line, checked before they are published, and can be uploaded by SFTP, FTP or FTPS.

• Admin demo: https://demo.commercegears.com/try-admin/data-feed-manager
• Sample feeds on the demo store: https://demo.commercegears.com/media/feeds/google_shopping.xml and https://demo.commercegears.com/media/feeds/meta_catalogue.csv
• Changelog: Release notes · Questions: FAQ

1. Requirements

2. Installation

Install with Composer. The access keys come from your account; see https://commercegears.com/composer-access/ for the details.

composer config repositories.commercegears composer https://repo.commercegears.com
composer config --auth http-basic.repo.commercegears.com PUBLIC_KEY PRIVATE_KEY
composer require commercegears/module-data-feed-manager
bin/magento setup:upgrade
bin/magento setup:di:compile            # production mode only
bin/magento setup:static-content:deploy # production mode only
bin/magento cache:flush

The installation imports Google's product taxonomy in English (en-GB). Make sure Magento's cron is running (bin/magento cron:install) for scheduled feeds. To update later, run composer update commercegears/module-data-feed-manager and the same bin/magento commands.

3. Settings

Stores > Configuration > CommerceGears > Data Feed Manager (also under CommerceGears > Data Feed Manager > Settings). Store view, currency and schedule are set per feed; these settings apply to all feeds.

4. Creating a feed

Go to CommerceGears > Data Feed Manager > Feeds and open the Add Feed split button. Choose From template: Google Shopping, Meta catalogue, Pinterest catalogue, Microsoft/Bing Shopping or generic JSON, or Blank feed. The main part of the button opens a blank feed. The feed form has tabs on the left.

General

Content: template (XML and JSON)

The editor highlights the template so the parts are easy to tell apart: template tags and keywords, variables, filters, strings and comments, and the XML (or JSON) around them. Line numbers are on the left and the cursor's line and column below the field. An unclosed {{, {% or {# is marked in red. Tab indents (several selected lines at once), Shift+Tab outdents; to leave the field with the keyboard press Esc, then Tab. Undo (Ctrl+Z) works as usual, and the template is saved exactly as typed.

{% for product in products %}
<item>
  <g:id>{{ product.sku }}</g:id>
  <g:title>{{ product.name | strip_tags | clean | truncate(150, '') }}</g:title>
  <g:price>{{ product.regular_price_incl_tax | price }}</g:price>
  {% if product.on_sale %}<g:sale_price>{{ product.final_price_incl_tax | price }}</g:sale_price>{% endif %}
</item>
{% endfor %}

• {{ … }} prints a value, escaped for XML or JSON automatically.
• {% for product in products %} … {% endfor %} marks the repeated item (exactly one, at the top level). Everything before it is the header, everything after it the footer.
• {% if %}, {% elseif %}, {% else %}, {% for image in product.images %}, {% set x = … %} and {# comments #}; loop.index, loop.first and loop.last inside loops.
• Operators: and or not == != < > <= >= in ~ + - * /, the fallback a ?? b and a ? b : c.
• Filters include strip_tags, truncate, upper, lower, capitalize, title, trim, clean, number_format, round, price, default, replace, regex_replace, prepend, append, join, split, first, last, slice, count, unique, date, html_decode, url_encode, json, cdata, raw and escape.

Variables and Filters below the editor lists every variable with a description: product attributes (store view values and option labels, product.raw.<code> for the stored value), prices (regular_price, final_price, special_price, min_price, max_price, each also _incl_tax and _excl_tax, tier_prices, on_sale), stock (qty, is_in_stock, is_salable, availability), url, images (image, images, all_images and more), categories (category, category_path, google_category_id, google_category), brand, parent.* and item_group_id for variants, variants and children, reviews (reviews_count, rating), store.*, feed.* and now.

The language is a sandbox: it has no access to PHP, files or configuration. For safety, XML feeds may not contain XHTML, SVG, MathML, XSLT or XLink content or an xml-stylesheet instruction.

Content: columns (CSV and TXT)

Each column has a Header, a Value (the same template syntax, or static text such as new) and a Default (when empty). Drag the handle to reorder columns. Below the columns: Delimiter (comma, semicolon, tab, pipe), Enclosure (double quote, single quote, none), Encoding (UTF-8, UTF-8 with BOM for Excel, ISO-8859-1, Windows-1252), Line Ending (LF or CRLF) and Header Row.

Product Filters

Schedule

Generate: Manual only (admin or CLI), Daily at set times (for example 03:00, 15:00) or a Cron expression (for example 0 */4 * * *), in the store view's timezone. The form shows the next run.

Delivery (FTP / SFTP)

Upload After Generation uploads the file after each generation; the local file is always written too. Fields: Protocol (SFTP, FTP, FTPS with explicit TLS), Host, Port (empty = 22 for SFTP, 21 for FTP and FTPS), Username, Password (stored encrypted), Remote Directory (must exist), SFTP Host Key and Passive Mode (FTP). Test Connection checks the settings in the form.

• SFTP host key check: the server's key fingerprint (SHA256:…) is saved on the first upload, or you enter it. If the server later presents another key, the upload stops before the password is sent. Clear the field when the server was really replaced.
• After changing the host or protocol, enter the password again; the saved password is not sent to another host.
• A failed upload marks the run as a warning; the local file is still updated.
• Tested status: FTP upload was tested against a local FTP server. SFTP has not yet been tested against a real SFTP server. FTPS does not verify the server's TLS certificate (a limit of PHP's FTP extension), so prefer SFTP.

Preview

Click Preview in the page header (or Preview Output above the template or columns) at any time: a panel shows the feed for the first matching products, rendered from the values in the form, including unsaved changes, and validates it. The store view, product filters and attribute conditions of the feed apply. The output is highlighted (XML, JSON, or CSV/TXT with the delimiters marked) and has line numbers. Nothing is saved or written. The Preview tab of the form shows the same with Preview First Items.

• How many items: the Preview Items setting (default 5, at most 20). The summary also says how many products match the filters. Products are counted once: when configurable products are exported as their variants, the summary says how many of the matching products are configurable, and the feed has more items than matching products. For a saved feed it also gives the item count of the last generated file.
• "Valid file format" means the output parses (XML, JSON, CSV/TXT columns). It does not check a channel's own rules: for example Google assigns a category itself when g:google_product_category is missing, and a product without categories has no product_type.
• A template error names its line; Show line N in the template opens the Content tab with the cursor on it.
• Items that fail (for example invalid XML) are listed with the SKU and the line of the item's output; a validation error of the whole output marks that line in the output.
• When the template prints nothing for most products, the preview stops after reading 500 products (or about 10 seconds) and says so.

Saving

Save keeps you on the form; the split button also has Save & Close and Save & Generate Now. Invalid file locations and template syntax errors are refused on save, and the entered values are kept.

5. Generating feeds

• By cron: feeds with a schedule are generated in the cron group commercegears_datafeed.
• In the admin: Generate Now (in the form or the grid's Select menu) runs batch by batch in the browser with a progress bar, then shows items, errors, file size, duration and the URL.
• On the command line: see section 9.

Each run writes to a temporary file. Items that fail their check are skipped and logged. The finished file is validated (streamed XML parse, JSON decode) and only then replaces the old file, in one step. If the file is invalid, it is not published and the previous file stays. Two runs of the same feed cannot overlap.

The grid shows each feed's file link, schedule, next run, last generated time and result (add the Items and Size columns under Columns), and has a keyword search. Public feeds show their URL in the File column; feeds below var/ show their location relative to the Magento root. Mass actions: enable, disable and delete. Row actions: Edit, Generate Now, View Log, Duplicate and Delete; Generate Now, Duplicate and Delete ask for confirmation first. Deleting a feed keeps its generated file: the message names the file, so you can delete it yourself if channels should no longer read it.

CommerceGears > Data Feed Manager > Generation Log lists every run: feed, start time, started by (cron, admin or CLI), status, seconds, items, errors, file size and messages. A run that never finished (for example the server restarted, or the browser tab was closed during Generate Now) is marked as an error, counted as one error, by the daily log clean-up (03:17) once it is more than 75 minutes old.

6. Google category mapping

CommerceGears > Data Feed Manager > Google Category Mapping lists your store categories. Type in the field next to a category (for example "hoodies") and pick a Google category. A product uses the mapping of its deepest category; categories without a mapping inherit the nearest mapped parent (shown in grey). Use Find category to filter the list and Save Mapping to save. In templates use {{ product.google_category_id }} or {{ product.google_category }}. The mapping applies to all feeds and store views.

Taxonomy Language: download taxonomy-with-ids.<locale>.txt from Google (for example taxonomy-with-ids.de-DE.txt) and import it at the bottom of the page or with commercegears:feed:taxonomy-import. Category IDs are the same in every language, so existing mappings stay valid.

7. File locations and privacy

• Feeds can be written below pub/ (public; below pub/media/ the URL is the media URL) or below var/ (private). pub/static/, pub/errors/ and hidden folders are not allowed.
• File names may only contain letters, digits, ., - and _.
• Outside pub/media/feeds/ a feed can only create a new file: a location where another file already exists (for example pub/robots.txt) is refused.
• Public files can be read by anyone who has the URL. Put feeds with private data (cost, stock levels) below var/ and upload them by SFTP.

8. Permissions (ACL)

Under System > Permissions > User Roles > Role Resources > CommerceGears > Data Feed Manager:

9. Command line

bin/magento commercegears:feed:list                                   # feeds with schedule, last run, items, status and URL
bin/magento commercegears:feed:generate <id> [<id> ...]               # generate the given feeds
bin/magento commercegears:feed:generate --all                         # generate all enabled feeds
bin/magento commercegears:feed:preview <id> [--limit=3]               # print the first items
bin/magento commercegears:feed:taxonomy-import [path/to/taxonomy-with-ids.de-DE.txt]

10. Troubleshooting

11. Limitations

• Prices are guest prices from the price index; tax uses the store's default tax destination.
• Attribute conditions apply to the selected products, not to variants added through the Configurable Products mode.
• The Google category mapping is global, not per store view or feed.
• CSV values are written as they are, without protection against spreadsheet formulas; do not open feeds built from untrusted text in a spreadsheet program.
• SFTP upload has not yet been tested against a real SFTP server; FTPS does not verify TLS certificates.

12. Uninstall

bin/magento module:disable CommerceGears_DataFeed
composer remove commercegears/module-data-feed-manager
bin/magento setup:upgrade

Feed files that were already generated stay where they are; delete them yourself if channels should no longer read them.